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EXECUTIVE  SUMMARY 


The  performance  of  man-portable  and  hand-held  configurations  of  the  Naval  Research 
Laboratory  TEMTADS  sensor  are  presented.  Both  systems  are  based  on  the  transient 
electromagnetic  induction  (TEM)  sensor  technology  that  was  developed  under  Environmental 
Security  Technology  Certification  Program  project  MR-200601,  “EMI  Array  for  Cued  UXO 
Discrimination,”  or  the  TEMTADS  5x5  array.  The  man-portable  (MP)  system  was  constructed 
as  a  2x2  array  of  the  sensors  developed  for  the  original  TEMTADS.  Eor  the  hand-held  (HH) 
sensor,  a  single,  coaxial  Tx/Rx  coil  pair  was  developed  to  capture  the  performance  of  the  original 
sensor  while  made  rugged  enough  for  handheld  use  in  the  field.  The  required  data  diversity  for 
the  HH  sensor  comes  from  making  a  series  of  40  measurements  over  the  target  using  a  physical 
template  for  precise  relative  geolocation.  Both  systems  are  designed  to  be  deployable  in 
increasingly  inaccessible  areas  where  vehicle-towed  sensor  arrays  cannot  be  used. 

Demonstrations  of  these  systems  have  been  conducted  at  our  test  facility  at  Blossom  Point,  MD, 
at  the  UXO  Standardized  Test  Site  at  Aberdeen  Proving  Ground,  and  at  live  sites  in  Bridgeport, 
CT  and  Washington,  DC.  These  sites  offer  a  range  of  UXO  sizes  and  types  along  with  a 
selection  of  munitions-related  scrap  and  cultural  clutter.  The  results  of  these  demonstrations  are 
discussed  in  terms  of  classification  performance  and  production  rate. 

Eor  the  MP  system,  the  APG  results  indicated  that  the  inversion  performance  of  the  system  was 
not  comparable  to  that  of  the  full  TEMTADS  5x5  array  for  lower  SNR  targets  due  to  the  limits 
of  the  smaller  data  set  (fewer  looks  at  the  target).  The  results  of  the  live  site  demonstrations 
supported  the  conclusions  drawn  after  the  APG  demonstration. 

Revision  of  the  sensor  technology  was  indicated  for  the  MP  system  to  collect  sufficient  data  over 
an  anomaly.  A  modified  version  of  the  EMI  sensors  in  the  MP  system  was  designed  and  built, 
replacing  the  single,  vertical-axes  receiver  loops  of  the  original  coils  with  three-axis  receiver 
cubes.  The  new  sensor  elements  were  designed  to  have  the  same  form  factor  as  the  originals, 
aiding  in  system  fabrication. 

The  HH  sensor  was  designed  for  use  in  extremely  limiting  terrain  and  for  integration  with  unique 
positioning  technologies.  The  APG  results  for  the  HH  sensor  indicated  that  the  inversion 
performance  of  the  system  using  a  36-point  observation  grid  was  comparable  to  that  of  the  full 
TEMTADS  5x5  array. 

The  primary  performance  comparisons  are  referenced  against  the  performance  of  the  original 
TEMTADS  5x5  array. 
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1.0  INTRODUCTION 


1.1  BACKGROUND 

Unexploded  ordnance  (UXO)  contamination  at  former  and  current  Department  of 
Defense  sites  is  an  extensive  problem.  Site  characterization  and  remediation  activities 
conducted  with  the  current  state-of-the-art  technologies  at  these  sites  often  yield 
unsatisfactory  results  and  are  extremely  expensive  to  implement.  This  is  due  in  part  to  the 
inability  of  current  technology  to  distinguish  between  UXO  and  nonhazardous  items. 
Newly-emerging  electromagnetic  induction  (EMI)  sensor  technologies  offer  the  ability  to 
robustly  distinguish  between  these  two  classes  of  objects.  Early  versions  of  these 
systems  have  tended  to  be  large  and  designed  for  towed  operation  on  open  fields  with 
good  sky  view  to  provide  the  necessary  quality  of  geolocation  information.  The  objective 
of  Environmental  Security  Technology  Certification  Program  (ESTCP)  projects  MR- 
200807  and  MR-200909  is  to  demonstrate  sensor  arrays  that  are  capable  of  reliably 
retaining  the  performance  of  one  of  these  new  technologies  in  a  form  suitable  for  use  in 
rugged  terrain  and  other  environments  where  mobility  and  the  viability  of  traditional 
positioning  technologies  are  limited.  The  systems  demonstrated  in  both  projects  are  based 
on  the  transient  electromagnetic  induction  (TEM)  sensor  technology  that  was  developed 
under  ESTCP  project  MR-200601. 

1.2  OBJECTIVES  OF  THE  PROJECTS 

The  objective  of  these  ESTCP-funded  Naval  Research  Eaboratory  (NRL)  projects  was  to 
validate  new  UXO  classification  technologies  through  a  series  of  blind  test 
demonstrations.  We  have  conducted  frequent  shake-down  demonstrations  of  each 
technology  at  our  Blossom  Point,  MD  field  site  but  a  blind  test  is  the  only  true  measure  of 
system  performance.  Both  sensor  technologies  were  demonstrated  at  the  Aberdeen 
Proving  Ground  (APG)  Standardized  UXO  Test  Site.  The  TEMTADS  MP  2x2  Cart  (MP 
system)  was  also  demonstrated  at  the  DuPont  Remington  Woods,  CT  site  several  times 
during  development  as  part  of  an  ongoing  classification-based  UXO  remediation  effort. 
The  MP  system  array  conducted  a  brief,  exploratory  demonstration  at  the  Dalecarlia 
Woods  site  in  Spring  Valley,  Washington,  DC  with  sponsorship  from  the  U.S.  Army 
Corps  of  Engineers  (USAGE),  Huntsville  through  their  Innovative  Technologies 
Program. 

1.3  REGULATORY  DRIVERS 

Stakeholder  acceptance  of  the  use  of  classification  techniques  on  real  sites  will  require 
demonstration  that  these  techniques  can  be  deployed  efficiently  and  with  high  probability 
of  discrimination.  The  first  step  in  this  process  is  to  demonstrate  acceptable  performance 
on  synthetic  test  sites  such  as  that  at  Aberdeen.  As  a  second  step,  demonstration  in  more 
real-world  scenarios  is  required.  Further  demonstration  at  live  sites  with  more  extensive 
ground-truth  validation  will  further  facilitate  regulatory  acceptance  of  the  UXO 
classification  technology  and  methodology. 


Manuscript  approved  January  27,  2012. 
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2.0  TECHNOLOGY 


2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  EMI  Sensors 

Two  types  of  sensors  are  discussed  in  this  report.  The  first  is  the  EMI  sensor  developed 
for  the  NRL  TEMTADS  5x5  array  under  ESTCP  project  MR-200601  and  described  in 
the  next  paragraph,  consisting  of  a  single  35  cm  x  35cm  transmitter  loop  coaxially 
located  with  a  single  25  cm  x  25cm  vertical-axis  receiver  loop.  The  second  is  the 
‘TEMTADS/3D’  sensor  in  which  the  same  transmitter  coil  is  used  but  the  receiver  coil  is 
replaced  by  an  8  cm,  3-component  ‘cube’  receiver  that  was  first  developed  by  G&G 
Sciences  under  a  Navy-funded  project  known  as  the  Advanced  Ordnance  Locator  (AOL). 
We  have  adopted  systems  made  from  multiple  copies  of  these  sensors,  assembled  in  a 
variety  of  array  configurations.  We  also  made  minor  modifications  to  the  control  and 
data  acquisition  computer  to  make  it  compatible  with  our  deployment  schemes. 

A  photograph  of  a  standard  TEMTADS  sensor  element  (as  used  in  the  MR-200601  array) 
is  shown  under  construction  in  the  left  panel  of  Figure  2-1 .  The  Tx  coil  is  wound  around 
the  outer  portion  of  the  form  and  measures  35  cm  on  a  side.  The  25  cm  x  25  cm,  square 
Rx  coil  is  wound  around  the  inner  part  of  the  form  which  is  re-inserted  into  the  outer 
portion  with  the  Rx  coil  in  place.  An  assembled  sensor  with  the  top  and  bottom  caps 
used  to  locate  the  sensor  in  the  array  is  shown  in  the  right  panel  of  Figure  2-1. 


Figure  2-1  -  Construction  details  of  an  individual  standard  TEMTADS  EMI 
sensor  (left  panel)  and  the  assembled  sensor  with  end  caps  attached  (right  panel). 

Decay  data  are  collected  with  a  500  kHz  sample  rate  until  25  ms  after  turn  off  of  the 
excitation  pulse.  A  raw  decay  consists  of  12,500  points;  too  many  to  be  used  practically. 
These  raw  decay  measurements  are  grouped  into  122  logarithmically-spaced  “gates”  with 
center  times  ranging  from  25  qs  to  24.375  ms  with  5%  widths  and  the  biimed  values  are 
saved  to  disk. 
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2.1.2  TEMTADS  Hand-Held  EMI  Sensor 


For  the  TEMTADS  Hand-Held  sensor  (HH  sensor),  a  new  configuration  of  the 
TEMTADS  EMI  sensor  was  developed  that  is  rugged,  weather-proof,  and  designed  with 
the  needs  of  a  handheld  instrument  in  mind.  The  sensor  includes  a  35-cm  diameter  Tx 
coil  and  an  inner,  25-cm  diameter  Rx  coil.  The  assembled  coil  is  significantly  thinner 
than  the  TEMTADS  sensor  (2  vs.  8  cm)  and  is  designed  with  a  clear  center  aperture 
which  can  be  fitted  with  a  variety  of  alignment  fixtures.  Shown  in  Eigure  2-2  is  a  simple 
cross-hair  arrangement  made  from  clear  acrylic. 


Figure  2-2  -  Construction  details  of  the  TEMTADS  Hand-Held  Sensor  (left  panel)  and  the 
assembled  sensor  (right  panel). 

2.1.3  EMI  Sensor  with  Tri-axial  Receiver  Cubes 

As  will  be  discussed  further  in  Section  7.8,  after  demonstration  of  the  MP  system  at  the 
APG  Standardized  UXO  Test  Site  in  August,  2010  [1],  revision  of  the  sensor  technology 
was  indicated  for  the  MP  system  to  collect  sufficient  data  over  an  anomaly.  A  modified 
version  of  the  sensor  element  was  designed  and  built,  replacing  the  single,  vertical-axis 
receiver  coil  of  the  original  sensor  with  a  three-axis  receiver  cube.  These  receiver  cubes 
are  similar  in  design  to  those  used  in  the  second-generation  AOL  and  the  Geometries 
MetalMapper  (ESTCP  MR-200603)  system  with  dimensions  of  8  cm  rather  than  10  cm. 
The  CRREE  MPV2  system  (ESTCP  MR-201005)  uses  an  array  of  five  identical  receiver 
cubes  and  a  circular  transmitter  coil.  The  new  sensor  elements  are  designed  to  have  the 
same  form  factor  as  the  originals,  aiding  in  system  integration.  A  new  coil  under 
construction  is  shown  in  Eigure  2-3. 
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Figure  2-3  -  Individual  updated  TEMTADS  EMI  sensor 
with  3 -axis  receiver  under  construction. 

2,1,4  Application  of  the  Technology 

Application  of  this  technology  is  evaluated  for  ‘cued-target  classifieation.’  In  this 
application,  targets  have  been  previously  detected  and  a  list  of  target  positions  is 
developed  from  a  survey  by  some  other  geophysical  instrument;  for  example  an  EM61- 
MK2  eart  survey.  Each  target  location  is  marked  in  advance  with  a  non-metallic  pin  flag 
or  similar  item  and  preferably  labeled  with  the  target  ID  number.  The  operator  positions 
either  the  MP  system  or  a  wooden  template  for  the  HH  sensor  over  each  target  in  turn. 
Once  positioned  over  the  target,  data  acquisition  is  conducted.  Eor  the  HH,  a  monostatic 
measurement  is  made  at  each  marked  position  along  a  6x6  template  along  with  a  series  of 
background  measurements.  Eor  the  MP  array,  the  data  acquisition  computer  steps 
through  the  array  sensors  sequentially,  collecting  decays  from  all  twelve  receive  coils  for 
each  excitation.  These  data  are  then  inverted  for  target  location  and  characteristics. 

In  the  final  version  of  this  technology,  one  could  envision  the  inversion  being  performed 
on-the-fiy  prior  to  or  while  the  operator  moves  to  the  next  target.  Eor  these 
demonstrations,  we  performed  the  inversions  off-line  so  that  we  would  have  the  ability  to 
intervene  in  the  automatic  process  as  required.  The  TEM  data  were  transferred  to  the 
analyst  several  times  each  day  for  near  real-time  analysis  at  the  demonstration  site. 

2,2  TECHNOLOGY  DEVELOPMENT 

The  MP  system  is  a  man-portable  four-element  transient  EMI  system  designed  and  built 
by  the  NRL  with  funding  from  ESTCP,  to  transition  the  TEM  sensor  technology  of  the 
TEMTADS  towed  array  (ESTCP  Project  MR-200601)  to  a  more  eompact,  man-portable 
eonfiguration  for  use  in  more  limiting  terrain  under  project  MR-200909.  Like  the  towed 
array,  this  system  is  currently  configured  to  operate  in  a  cued  mode,  where  the  target 
location  is  already  known.  The  MP  system  is  shown  in  Eigure  2-4. 
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Figure  2-4  -  The  NRL  TEMTADS  MP  2x2  Cart 

Preliminary  testing  of  the  initial  system  configuration  [2]  found  that  for  high  SNR  (>  30) 
targets  one  measurement  cycle  provides  enough  information  to  support  classification.  For 
deeper  and/or  weaker  targets,  more  robust  estimates  of  target  parameters  are  obtained  by 
combining  two  closely-spaced  measurements.  Two  measurements  per  anomaly  were 
typically  made  proactively  to  avoid  the  potential  need  to  revisit  a  target  a  second  time  [2]. 
As  part  of  project  MR-200909,  a  demonstration  was  conducted  to  rigorously  investigate 
the  capabilities  of  this  new  sensor  platform  for  UXO  classification  in  a  cued  data 
collection  mode  at  the  APG  Standardized  UXO  Test  Site  in  August,  2010.  The  results 
are  presented  in  Section  7.0.  Those  results  indicated  that  the  inversion  performance  of 
the  system  was  not  comparable  to  that  of  the  full  TEMTADS  array  for  lower  SNR  targets 
due  to  the  limits  of  the  smaller  data  set  (fewer  looks  at  the  target). 

Several  additional  data  collection  windows  of  opportunities  were  available  prior  to  the 
APG  demonstration.  The  results  of  data  collection  efforts  at  the  Remington  Woods,  CT 
site  are  given  in  Section  7.6.  A  limited  amount  of  data  was  also  collected  at  the 
Dalecarlia  Woods,  Spring  Valley,  DC  site  and  the  results  are  given  in  Section  7.7.  The 
results  of  these  demonstrations  supported  the  conclusions  ultimately  drawn  after  the  APG 
demonstration. 

Revision  of  the  sensor  technology  was  indicated  for  the  MP  system  to  collect  sufficient 
data  over  an  anomaly.  A  modified  version  of  the  EMI  sensor  was  designed  and  built, 
replacing  the  single,  vertical  axis  receiver  loop  of  the  original  coil  with  a  tri-axial  receiver 
cube.  These  receiver  cubes  are  identical  in  design  to  those  used  in  the  CRREL  MPV2 
system  (ESTCP  MR-201005).  The  new  sensor  elements  were  designed  to  have  the  same 
form  factor  as  the  originals,  aiding  in  system  fabrication.  The  analyses  which  led  to  this 
design  selection  are  discussed  in  Section  7.8. 

The  TEMTADS  Hand-Held  Sensor  was  designed  to  package  the  TEMTADS  transient 
EMI  sensor  technology  into  a  more  compact,  hand-held  configuration  for  use  in 
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extremely  limiting  terrain  and  for  integration  with  unique  positioning  technologies  such 
as  the  SAINT  INS  demonstrated  under  ESTCP  project  MR-200810.  The  Hand-Held 
sensor  is  currently  configured  to  operate  in  a  cued  mode,  where  the  target  location  is 
already  known.  A  series  of  approximately  40  monostatic  measurements  are  made  over 
each  target  using  a  template  for  precise,  relative  geo-location.  The  TEMTADS  Hand- 
Held  sensor  is  shown  in  Eigure  2-5. 


Figure  2-5  -  The  NRL  TEMTADS  Hand-Held  Sensor 

2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  original  TEMTADS  5x5  array  was  designed  to  combine  the  data  advantages  of  a 
gridded  survey  with  the  coverage  efficiencies  of  a  vehicular  system.  The  MP  system  was 
designed  to  offer  similar  production  rates  in  difficult  terrain  and  treed  areas  that  the 
TEMTADS  5x5  array  cannot  access.  With  the  upgraded  EMI  sensors  which  incorporate 
the  tri-axial  receiver  cubes,  similar  performance  can  be  achieved  with  similar 
classification-grade  data  quality. 

The  MP  array  is  80  cm  x  80  cm,  square  and  mounted  on  a  man-portable  cart.  Terrain 
where  the  vegetation  or  topography  interferes  with  passage  of  a  cart  of  that  size  will  not 
be  amenable  to  the  use  of  the  system.  Eor  increasingly-difficult  survey  conditions,  the 
HH  system  allows  for  data  set  to  be  built  up  one  monostatic  element  at  a  time  for  flexible 
data  collection  geometries.  As  only  monostatic  measurements  can  be  made  with  the  HH, 
significantly  more  measurements  are  necessary,  reducing  the  production  rate. 

3.0  PERFORMANCE  OBJECTIVES 

Performance  objectives  for  the  demonstrations  are  given  as  a  basis  for  the  evaluation  of 
the  performance  and  costs  of  the  demonstrated  technologies.  Since  these  are 
classification  technologies,  the  performance  objectives  focus  on  the  second  step  of  the 
UXO  remediation  problem;  that  of  target  classification  as  UXO,  clutter,  etc.  We  assume 
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that  the  anomalies  from  all  targets  of  interest  have  been  deteeted  and  have  been  ineluded 
on  the  target  list. 


These  performanee  objectives  apply  to  the  APG  demonstrations  of  each  system.  For  the 
Remington  Woods  and  Spring  Valley  demonstrations,  the  MP  system  was  invited  to 
participate  in  ongoing  remediation  efforts  without  formal  demonstration  plans.  The 
performances  at  each  site  are  discussed  in  Sections  7.6  and  7.7,  respectively. 

Table  3-1  -  Performance  Objectives  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Quantitative  Performance  Objectives 

Correct 

classification  of 
targets  of  interest 

Number  of  targets 
of  interest  identified 

•  Prioritized  dig  list 

•  Scoring  report 
from  APG 

95%  correct 
identification  of  all 
targets  of  interest 

Reduction  of  False 
Alarms 

Number  of  false 
alarms  eliminated 

•  Prioritized  dig  list 

•  Scoring  report 
from  APG 

Reduction  of  false 
alarms  by  50%  or 
more  with  95% 
correct  identification 
of  munitions 

Cued  Production 

Rate 

Number  of  cued 
targets  investigated 
per  day 

Log  of  field  work 

Hand-Held: 

50/day 

MP  2x2  Cart: 

200/day 

Analysis  Time 

Average  time 
required  for 
inversion  and 
classification 

Log  of  analysis 
work 

15  min  per  target 

Qualitative  Performance  Objectives 

Ease  of  Use 

System  can  be  used 
in  the  field  without 
significant  issues 

Team  feedback 

Lield  team  has  no 
significant  issues  to 
report 

Robustness  & 
Reliability 

•  Number  of 
operational  hours 
recorded  per  day 

•  Number  of 
significant 
technical  issues 

•  Lield  logs  of 
operational  hours 
per  day 

•  Lield  logs  of 
significant 
technical  issues 

•  >  6  hrs/day 

•  <  1  significant 
technical  issue  per 
day 
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3.1  CORRECT  CLASSIFICATION  OF  TARGETS  OF  INTEREST 


This  is  one  of  the  two  primary  measures  of  the  elassifieation  value  of  the  data  eollected 
by  these  sensor  systems.  By  eollecting  high-quality,  preeisely  relatively-loeated  data,  it 
should  be  possible  to  diseriminate  munitions  from  scrap  and  frag  with  some  efficiency. 
We  expected  to  properly  classify  a  large  percentage  of  the  seeded  munitions  items. 

3.1.1  Metric 

At  a  seeded  test  site  such  as  the  APG  Standardized  UXO  Test  Site,  the  metric  for 
classification  efficiency  is  straightforward.  We  prepared  a  ranked  dig  list  from  the  survey 
data  with  a  UXO  /  Clutter  decision  for  each  Blind  Grid  cell  and  for  each  location  in  the 
Indirect  Fire  Area  that  the  MP  2x2  Cart  investigated.  ATC  personnel  used  their 
automated  scoring  algorithms  to  assess  our  results. 

3.1.2  Data  Requirements 

The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators. 
Our  ranked  dig  lists  were  the  input  for  this  metric  and  ATC’s  standard  scoring  was  the 
output. 


3.1.3  Success  Criteria 

The  objective  was  considered  to  be  met  for  each  demonstration  if  more  than  95%  of  the 
seeded  munitions  items  were  correctly  classified. 

3.2  OBJECTIVE:  REDUCTION  OF  FALSE  ALARMS 

This  is  the  second  of  the  two  primary  measures  of  the  classification  value  of  the  data 
collected  by  these  technologies.  By  collecting  high-quality,  precisely  relatively-located 
data,  it  should  be  possible  to  discriminate  munitions  from  scrap  and  frag  with  some 
efficiency.  We  expected  to  properly  classify  a  large  percentage  of  the  clutter  as  such. 

3.2.1  Metric 

At  a  seeded  test  site  such  as  the  APG  Standardized  UXO  Test  Site,  the  metric  for  false 
alarm  elimination  is  straightforward.  We  prepared  a  ranked  dig  list  from  the  survey  data 
with  a  UXO  /  Clutter  decision  for  each  Blind  Grid  cell  and  for  each  location  in  the 
Indirect  Fire  Area  that  the  MP  system  investigated.  ATC  personnel  used  their  automated 
scoring  algorithms  to  assess  our  results. 

3.2.2  Data  Requirements 

The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators. 
Our  ranked  dig  lists  were  the  input  for  this  metric  and  ATC’s  standard  scoring  was  the 
output. 
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3,2.3  Success  Criteria 


The  objective  was  considered  met  if  more  than  50%  of  the  non-munitions  items  were 
labeled  as  no-dig  while  retaining  95%  of  the  munitions  items  on  the  dig  list. 

3.3  OBJECTIVE:  CUED  PRODUCTION  RATE 

Even  if  the  performance  of  the  technologies  on  the  metrics  above  was  satisfactory,  there 
remain  economic  metrics  to  consider.  Survey  efficiency  is  the  metric  that  was  tracked  in 
these  demonstrations. 

3.3.1  Metric 

For  cued  data  collection,  the  metric  is  the  number  of  anomalies  investigated  per  day 
during  each  demonstration.  Combined  with  the  daily  operating  cost  of  the  technology, 
these  values  give  the  per-anomaly  cost  of  operating  each  technology. 

3.3.2  Data  Requirements 

Productivity  was  determined  from  a  review  of  the  demonstration  field  logs. 

3.3.3  Success  Criteria 

Given  the  cued  data-collection  methodology  used  for  these  demonstrations,  this  objective 
was  considered  successfully  met  if  the  production  rates  were  at  least  50  and  200 
anomalies  per  day  for  the  Hand-Held  sensor  and  the  MP  system,  respectively. 

3.4  OBJECTIVE:  ANALYSIS  TIME 

Another  component  of  demonstration  costs  was  the  amount  of  analyst  time  required  for 
data  analysis.  We  tracked  the  near-real-time  analysis  time  for  these  demonstrations. 

3.4.1  Metric 

The  time  required  for  inversion  and  classification  per  anomaly  was  the  metric  for  this 
objective. 

3.4.2  Data  Requirements 

Analysis  time  was  determined  from  a  review  of  the  data  analysis  logs. 

3.4.3  Success  Criteria 

Since  these  were  the  first  formal  demonstrations  of  these  technologies,  the  objective  was 
considered  successfully  met  if  the  average  inversion  and  classification  time  was  less  than 
15  minutes  per  anomaly. 
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3.5  OBJECTIVE:  EASE  OF  USE 


This  objective  represents  an  opportunity  for  all  parties  involved  in  the  data  collection 
process,  especially  the  data  collection  team,  to  provide  feedback  in  areas  where  the 
process  could  be  improved. 

3.5.1  Data  Requirements 

Discussions  with  the  entire  field  team  and  other  observations  were  used. 

3.6  OBJECTIVE:  RELIABILITY 

This  objective  captures  the  readiness  of  the  system  for  live  site  demonstrations  as  an 
integrated  system. 

3.6.1  Data  Requirements 

The  number  of  operational  hours  per  day  and  the  frequency  of  significant  technical  issues 
were  collected  from  the  demonstration  field  logs. 

4.0  SITE  DESCRIPTION 

For  each  of  these  projects  one  demonstration  was  conducted  at  the  APG  Standardized 
UXO  Test  Site  located  at  the  Aberdeen  Proving  Ground,  MD.  The  MP  system  was 
demonstrated  in  August,  2010  and  the  HH  sensor  was  demonstrated  in  October,  2010. 

The  site  description  for  APG  is  given  in  Section  4.1.  The  MP  system  participated  in  a 
pair  of  small-scale  demonstrations  at  the  Remington  Woods  site  in  October,  2008  and 
August,  2009.  The  Remington  Woods  site  is  discussed  in  Section  4.2.  In  May,  2010,  the 
MP  system  made  measurements  on  107  anomalies  in  the  Dalecarlia  Woods  site.  A  brief 
discussion  of  the  Spring  Valley  site  is  provided  in  Section  4.3. 

4.1  APG  Standardized  UXO  Test  Site 

4.1.1  Site  Selection 

APG  was  initially  chosen  as  the  site  of  the  first  field  demonstration  for  each  technology. 
The  APG  site  is  located  close  to  our  base  of  operations  in  southern  Maryland  and 
therefore  minimizes  the  logistics  costs  of  deployment.  Use  of  this  site  allows  us  to 
receive  validation  results  from  near-real-world  conditions  without  incurring  the  logistics 
and  intrusive  investigation  expenses  that  would  be  required  for  a  demonstration  at  a  live 
site. 

4.1.2  Site  History 

The  Standardized  UXO  Test  Site  is  adjacent  to  the  Trench  Warfare  facility  at  the 
Aberdeen  Proving  Ground.  The  specific  area  was  used  for  a  variety  of  ordnance  tests 
over  the  years.  Initial  magnetometer  and  EMI  surveys  conducted  by  the  MTADS  team 
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performed  after  a  “mag  and  flag”  survey  of  the  same  area  identified  over  a  thousand 
remaining  anomalies.  These  data  were  used  for  a  final  cleanup  of  the  site  prior  to  the 
emplacement  of  the  original  test  items.  Prior  to  the  two  subsequent  reconfiguration 
events,  unexplained  anomalies  identified  by  demonstrators  using  the  site  were  also 
investigated  and  removed. 

4.1.3  Site  Topography  and  Geology 

According  to  the  soils  survey  conducted  for  the  entire  area  of  APG  in  1998,  the  test  site 
consists  primarily  of  Elkton  Series  type  soil  [3].  The  Elkton  Series  consist  of  very  deep, 
slowly  permeable,  poorly  drained  soils.  These  soils  formed  in  silty  aeolin  sediments  and 
the  underlying  loamy  alluvial  and  marine  sediments.  They  are  on  upland  and  lowland 
flats  and  in  depressions  of  the  Mid-Atlantic  Coastal  Plain.  Slopes  range  from  0  to  2 
percent. 

Overall,  the  demonstration  site  is  relatively  flat  and  level.  There  are  some  low-lying 
areas  in  the  northwest  portion  of  the  site  that  tend  to  have  standing  water  during  the  wet 
periods  of  the  year.  The  current  sensor  systems  are  moderately  weatherproofed,  but  we 
did  not  operate  them  through  standing  water.  However,  during  the  most  recent 
reconfiguration,  the  areas  most  prone  to  being  underwater  were  excluded  from  the  survey 
scenarios.  Anomalies  that  were  located  underwater  or  nearby  to  water  at  the  time  of 
survey  were  deferred  until  the  end  of  the  survey  and  were  interrogated  by  carefully,  if 
less  efficiently,  maneuvering  the  array  into  position.  A  small  number  of  the  Calibration 
Area  items  remained  under  a  sufficient  depth  of  water  to  be  rendered  inaccessible  to  the 
HH  sensor  throughout  the  demonstration. 

4.1.4  Munitions  Contamination 

The  area  currently  occupied  by  the  UXO  Site  has  seen  an  extensive  history  of  munitions 
use.  As  an  example,  in  2003  we  conducted  a  magnetometer  survey  of  a  previously 
unremediated  area  directly  adjacent  to  the  site  [4].  In  a  survey  area  of  approximately  1 
hectare,  we  identified  2,479  anomalies,  of  which  1,921  were  amenable  to  a  model  fit 
using  our  standard  analysis.  Historical  records  provided  by  ATC  and  previous 
remediation  results  indicated  that  the  likely  munitions  of  interest  for  this  site  were: 

•  Grenades,  Mkl,  Mkll,  and  Erench  VB  Rifle  w/o  chute 

•  Grenades,  Erench  VB  Rifle  w/  chute 

•  60mm  mortars  (including  2”  Smoke) 

•  3”  Stokes  (Smoke  and  HE) 

•  105  mm  projectiles 

•  155  mm  projectiles 
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4,1.5  Site  Geodetic  Control  Information 


There  are  two  first-order  points  on  the  site  for  use  as  GPS  base  station  points.  Their 
reported  coordinates  are  listed  in  Table  4-1.  The  horizontal  datum  for  all  values  is 
NAD83.  The  vertical  control  is  referenced  to  the  NAVD88  datum  and  the  GeoidOS 
geoid.  All  anomaly  list  locations  for  the  APG  demonstrations  were  flagged  by  APG 
geodetics  personnel  using  their  standard  techniques. 


Table  4-1  -  Geodetic  Control  at  the  APG  Standardized  UXO  Test  Site 


ID 

Latitude 

Longitude 

Elevation 

Northing 

Easting 

HAE 

477 

39°  28'  18.63880"  N 

76°  07'  47.71815"W 

10.669  m 

4,369,749.013 

402,810.038 

-22.545 

478 

39°  28' 04.24219"  N 

76°  07'  48.50439"W 

11.747  m 

4,369,305.416 

402,785.686 

-21.473 
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4,1,6  Site  Configuration 


Figure  4-1  is  a  map  of  the  Standardized  UXO  Technology  Demonstration  Site  at  APG. 
The  Calibration  and  Blind  Grids  are  shown  along  with  the  various  Open  Field  Areas. 


Figure  4-1  -  Map  of  the  reeonfigured  APG  Standardized  UXO  Test  Site. 
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4,2  Remington  Woods,  CT 
4,2,1  Site  Selection 


The  Remington  Woods  site  in  Bridgeport,  CT  is  part  of  the  former  Remington  Arms 
Lake  Suecess  property.  SAIC  has  been  supporting  the  UXO  remediation  efforts  of  the 
current  property  owners,  DuPont  Corporation,  for  a  number  of  years.  Advanced  data 
collection  and  data  analysis  techniques  have  been  applied  to  expedite  the  remediation 
effort  on  site  with  the  support  of  all  stakeholders.  Based  on  discussions  with  the  site 
team,  the  MP  system  was  invited  to  conduct  side-by-side  data  collection  and  analysis 
efforts  with  the  current  process  (which  will  be  described  in  more  detail  in  Section  7.6) 
where  the  MP  system  would  collect  data  over  selected  flags  already  placed  for 
remediation  by  the  current  process.  The  data  would  be  analyzed  and  a  dig  list  prepared. 
Since  these  flags  were  already  scheduled  for  remediation,  ground  truth  was  available  for 
determining  the  system  performance. 

4.2.2  Site  History 

The  site  was  used  by  Remington  Arms  until  1989  for  production,  testing,  storage,  and 
disposal  of  small  and  large  caliber  ammunition  and  powders.  DuPont  Corporation  and 
URS  Corporation  have  been  working  to  clean  up  the  site  since  2002. 

4.2.3  Site  Topography  and  Geology 

The  site  is  mostly  covered  with  tall  trees  and  dense  thorny  underbrush.  The  underbrush  is 
cleared  from  the  current  work  area  each  year  to  increase  productivity.  Many  large  rock 
formations  are  distributed  over  the  site  as  well.  Several  structures  related  to  the  testing 
and  development  of  munitions  are  scattered  throughout  the  site.  A  25-acre  lake  is  located 
at  the  center  of  the  site. 


Figure  4-2  -  A  view  of  the  Remington  Woods  site. 
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4.2.4  Munitions  Contamination 


Munitions  items  found  at  the  site  tend  to  be  small  (37mm  -  57mm)  and  shallow  (the  depth 
distribution  of  exeavated  “potential  UXO”  contacts  is  shown  in  the  plot  below) 


Figure  4-3  -  Depth  distribution  of  UXO  found  at  the 
Remington  Woods  site. 

4.2.5  Site  Geodetic  Control  Information 

The  site  team  placed  all  target  flags  using  their  standard  techniques.  Therefore  no 
information  regarding  geodetic  control  was  provided  to  us. 

4.2.6  Site  Configuration 

The  422-acre  site  is  located  in  Bridgeport  and  Stratford,  CT.  A  25-acre  lake  is  located  at 
the  center  which  was  not  part  of  these  demonstrations.  Each  year’s  efforts  are  focused  on 
a  40  to  80-acre  subarea.  A  schematic  map  of  the  site  is  shown  in  Figure  4-4. 
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URS 

URSCcrporatiofi 
Iron  Hill  Corporate  Center 
4051  Ogletown  Road,  Suite  300 
Newark,  Delaware  19713 


201 1  UXO  Verif  i  ation  Area  with 
UXO  Locations 

18984345 

0225J011 

LAKE  SUCCESS  BUSINESS  PARK 

61 5  Asylum  Street 

Bridgeport,  Connecticut  06610 

1 

Figure  4-4  -  Map  of  the  Remington  Woods,  CT. 

4,3  Dalecarlia  Woods,  Washington,  DC 

4.3.1  Site  Selection 

The  U.S.  Army  Corps  of  Engineers,  Huntsville  has  an  Innovative  Technologies  Program 
for  investing  new  UXO  technologies  for  use  by  the  community.  Under  funding  from  this 
program,  the  Dalecarlia  Woods  site  was  selected  because  of  its  proximity  to  NRL,  SAIC- 
ASAD,  and  the  MTADS  home  facilities.  This  effort  was  conducted  with  cooperation 
from  the  USACoE,  Baltimore  district. 

4.3.2  Site  History 

The  Spring  Valley  Eormerly  Used  Defense  Site  (EUDS)  consists  of  approximately  661 
acres  in  the  northwest  section  of  Washington,  DC  [5].  During  the  World  War  I  era,  the 
site  was  known  as  the  American  University  Experiment  Station  (AUES),  and  was  used  by 
the  U.S.  Government  for  research  and  testing  of  chemical  agents,  equipment  and 
munitions.  Today,  the  Spring  Valley  neighborhood  encompasses  approximately  1,200 
private  homes,  including  several  embassies  and  foreign  properties,  as  well  as  the 
American  University  and  Wesley  Seminary. 

In  1993,  a  contractor  digging  a  utility  trench  in  Spring  Valley  discovered  buried 
munitions  (UXO)  [6].  141  items  (43  suspect  chemical  items)  were  removed  through  the 
resulting  emergency  response.  In  Eebruary  1993,  the  USAGE  began  to  conduct  a 
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remedial  investigation  of  the  site.  The  investigation  by  the  USAGE  focused  on  specific 
sites  that  were  determined  to  have  the  potential  for  contamination.  Following  a  two-year 
investigation,  the  USAGE  found  four  munitions  and  no  additional  chemical  warfare 
materiel.  In  1995,  a  No  Further  Action  Decision  Document  covering  most  of  the  site  was 
signed,  while  acknowledging  the  Army’s  responsibility  for  follow-up  action  if  needed. 

The  USAGE,  at  the  encouragement  of  the  regulatory  community,  returned  to  the  FUDS 
in  1998  and  conducted  further  investigation  on  the  residence  of  the  Ambassador  of  South 
Korea.  This  investigation  yielded  several  burial  pits  containing  munitions  items,  many  of 
which  were  filled  with  chemical  warfare  materiel.  The  USAGE  expanded  the 
investigation  to  include  every  property  located  in  the  FUDS  boundary.  This  investigation 
included  the  identification  and  removal  of  arsenic  contaminated  soil,  a  groundwater 
investigation,  and  the  search  for  additional  munitions,  both  in  burial  pits  and  isolated 
items  on  residential  properties. 

A  full  history  of  this  work  is  available  at  the  Spring  Valley  web  site  [5]. 

4.3,3  Site  Topography  and  Geology 

The  demonstration  site  is  primarily  gently  rolling  hills  with  some  steep  banks  at  the  edge 
of  streams.  There  is  moderate  tree  cover  on  the  site,  as  seen  in  Figure  4-5.  There  is 
significant  ground  cover  and  large,  fallen  trees  throughout  the  area. 


Figure  4-5  -  A  view  of  the  Dalecarlia  Woods  area 


4.3.4  Munitions  Contamination 

A  more  complete  discussion  of  the  munitions  and  munitions-related  materials  that  have 
been  found  within  the  FUDS  is  available  at  the  official  website  [5].  This  effort  was 
focused  on  75mm  projectiles,  4-in  Stokes  Mortars,  and  Fivens  Projectors. 
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4,3,5  Site  Geodetic  Control  Information 


The  site  team  plaeed  all  target  flags  using  their  standard  techniques.  Therefore  no 
information  regarding  geodetic  control  was  provided  to  us. 

4,3,6  Site  Configuration 

The  site  has  been  divided  into  a  series  of  sub-areas  (by  location)  and  grids  within  those 
areas.  Three  grids  were  made  available  in  the  Dalecarlia  Woods  area  for  the 
demonstration. 

5.0  TEST  DESIGN 

5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

Each  demonstration  was  designed  to  be  executed  in  two  stages.  The  first  stage  was  to 
characterize  the  response  of  the  sensor  system  with  respect  to  the  items  of  interest  and  to 
the  site-specific  geology.  Characterization  of  the  sensor  response  to  the  items  of  interest 
was  conducted  at  our  home  facility  using  both  test  stand  and  test  field  measurements 
prior  to  deployment.  The  background  response  of  the  demonstration  site,  as  measured  by 
the  sensor  systems,  was  characterized  throughout  data  collection.  If  any  items  of  interest 
were  only  available  onsite,  onsite  characterization  measurements  would  be  made  during 
the  demonstration. 

The  second  stage  of  each  demonstration  was  a  survey  of  the  demonstration  site  using  the 
specified  sensor  system.  The  system  (or  template)  was  positioned  roughly  over  the  center 
of  each  anomaly  on  the  source  anomaly  list  and  a  data  set  collected.  Each  data  set  was 
then  inverted  using  the  data  analysis  methodology  discussed  in  Section  6.0,  and  estimated 
target  parameters  determined. 

The  target  list  for  each  demonstration  was  developed  from  previously  acquired 
geophysical  data  analysis.  Eor  example,  for  the  MP  system  demonstration  at  APG,  the 
same  target  list  that  was  developed  for  the  TEMTADS  5x5  array  demonstration  at  APG 
was  used  to  provide  the  best  system-to-system  comparison.  The  data  collection  process 
is  described  in  more  detail  for  each  of  the  sensor  systems  in  the  following  two  sections. 

5.1.1  TEMTADS  Hand-Held  Sensor 

Eor  the  APG  demonstration  of  the  HH  sensor,  the  union  of  the  Blind  Grid  target  lists 
from  the  previous  EM61-HH  /  SAINT  and  TEMTADS  5x5  demonstrations  were  used  as 
the  target  list.  This  allowed  for  a  direct,  head-to-head  comparison  of  the  results  with  those 
of  the  EM61-HH  /  SAINT  configuration  and  comparisons  with  the  TEMTADS  5x5  array 
and  MP  system. 

A  wooden  template  was  positioned  over  each  target  in  turn.  A  series  of  40  individual 
measurements  was  then  made  using  the  template  as  a  precise  guide  for  relative  location. 
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For  each  measurement,  the  system  activated  the  transmitter  and  collected  decay  data  from 
the  Rx  coil.  The  sensor  was  then  moved  to  each  template  position  in  turn,  and  the  next 
set  of  data  was  collected.  In  addition  to  the  positions  on  the  template,  in-air  and  near¬ 
surface  background  locations  were  included  as  shown  schematically  in  Figure  5-1  b). 

The  position  numbering  on  the  schematic  indicates  the  recommended  order  of  collection. 
The  complete  set  of  data  for  each  target  was  then  inverted  for  target  characteristics. 
Complete  coverage  of  the  survey  areas  depended  on  weather  and  water  levels  in  sections 
with  poor  drainage. 


Figure  5-1  -  The  position  template  a)  over  a  test  article  and  b)  shown  schematically. 

5.1.2  TEMTADS  MP  2x2  Cart 

The  MP  system  was  positioned  roughly  centered  over  each  target  flag.  Once  positioned, 
data  were  collected  while  firing  each  transmitter  in  sequence.  In  previous  testing  [2],  we 
found  demonstrable  value  in  collecting  a  second  set  of  data  at  a  location  approximately 
20  cm  (1/2  a  sensor  width)  off  the  anomaly  center,  particularly  for  deeper  targets.  This 
process  was  continued  for  these  demonstrations.  Analyses  of  the  results  with  and  without 
this  second  data  set  were  included  in  our  assessment  of  the  performance  of  the  MP 
system.  See  Section  7.8  for  further  details.  The  anomaly  lists  for  the  Blind  Grid  and  the 
Indirect  Fire  Areas  were  the  same  ones  used  for  the  TEMTADS  5x5  array  demonstration 
in  June  2008  [7].  A  cued  data  collection  was  made  for  each  anomaly  position.  Complete 
coverage  of  these  areas  depended  on  weather  and  water  level  in  sections  with  poor 
drainage. 

5.2  SITE  PREPARATION 

Basic  facilities  such  as  portable  toilets  and  field  buildings  were  provided.  Secure  storage 
for  the  sensor  systems  was  available  in  the  field  buildings  on  site.  Site  personnel  placed 
plastic  pin  flags  with  the  flag  number  clearly  marked  at  each  flag  position  using  their 
standard  techniques  prior  to  each  demonstration. 

5.3  SYSTEMS  SPECIFICATION 

These  demonstrations  were  conducted  using  the  NRL  TEMTADS  Hand-Held  Sensor  and 
the  TEMTADS  MP  2x2  Cart. 
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5,3,1  TEMTADS  Array  Configuration  and  Electronics 


The  standard  TEMTADS  TEM  sensor  has  dimensions  of  40  em  x  40  em.  Sensors  are 
positioned  at  a  ride  height  of  10  -  20om  above  the  ground  to  minimize  the  effeets  of 
ground  response  while  maximizing  the  depth  of  targets  for  whieh  elassification  grade 
data  can  be  collected.  Each  sensor  platform  is  constructed  of  one  or  more  of  these 
sensors,  yielding  cross-track  and  down-track  separation  of  40  cm  for  array 
configurations.  The  transmitter  electronics  and  the  data  acquisition  computer  are 
mounted  in  the  operator  backpack,  as  shown  in  Eigure  5-2.  Custom  software  written  by 
NRL  provides  data  acquisition  functionality.  After  the  sensor/array  is  positioned  roughly 
centered  over  the  center  of  the  anomaly,  the  data  acquisition  cycle  is  initiated.  Each 
transmitter  is  fired  in  a  sequence.  The  received  signal  is  recorded  for  all  Rx  chaimels  for 
each  transmit  cycle.  The  transmit  pulse  waveform  duration  is  2.7  s  (0.9s  block  time,  9 
repeats  within  a  block,  3  blocks  stacked,  with  a  50%  duty  cycle).  While  it  is  possible  to 
record  the  entire  decay  transient  at  500  MHz,  we  have  found  that  binning  the  data  into 
122  time  gates  simplifies  the  analysis  and  provides  additional  signal  averaging  without 
significant  loss  of  temporal  resolution  in  the  transient  decays  [8].  The  data  are  recorded 
in  a  binary  format  as  a  single  file  with  four  data  points  (one  data  point  per  Tx  cycle).  The 
filename  corresponds  to  the  anomaly  ID  from  the  target  list  under  investigation. 


Figure  5-2  -  TEMTADS  2x2  Electronics  Backpack 

5,3,2  Data  Acquisition  User  Interface 

The  data  acquisition  computer  is  mounted  on  a  backpack  worn  by  one  of  the  data 
acquisition  operators.  The  second  operator  controls  the  data  collection  using  a  personal 
data  assistant  (PDA)  which  wirelessly  (IEEE  802.1  lb)  communicates  with  the  data 
acquisition  computer.  The  second  operator  also  manages  field  notes  and  team 
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orienteering  funetions.  Data  colleetion  with  the  MP  system  at  the  former  Camp  Beale, 
CA  is  shown  in  Figure  5-3. 


Figure  5-3  -  TEMTADS  MP  2x2  Cart  and  Data  Acquisition  Operators 

5.3.3  Hand-Held  TEMTADS  Sensor  System 

The  HH  sensor  is  deployed  on  a  raised  template  resulting  in  a  sensor-to-ground  offset  of 
up  to  25  em.  The  optimum  sensor  height  is  dependent  on  the  baekground  ground 
response  and  is  determined  on  a  site-by-site  basis.  The  HH  sensor  is  shown  in  Figure 
2-5.  At  this  point  in  the  projeet,  the  system  operates  in  a  eued  mode  only.  The  loeations 
of  the  anomalies  must  already  be  known  and  flagged  for  reaequisition.  In  the  future,  the 
system  will  be  equipped  with  GPS  and/or  other  positioning  systems. 

5.3.4  TEMTADS  MP  2x2  Cart 

The  MP  system  is  a  man-portable  system  eomprised  of  four  of  the  EMI  sensors 
developed  for  the  NRL  TEMTADS  5x5  array  arranged  in  a  2x2  array  as  shown 
sohematieally  in  Figure  5-4.  The  MP  system,  shown  in  Figure  5-5  at  APG,  is  fabrieated 
from  PVC  plastie  and  G-10  fiberglass.  The  eenter-to-eenter  distanee  is  40  em  yielding  an 
80  em  X  80  em  array.  The  array  is  deployed  on  a  set  of  wheels  resulting  in  a  sensor-to- 
ground  offset  of  approximately  25  cm.  At  this  point  in  the  project,  the  system  operates  in 
a  cued  mode  only.  The  locations  of  the  anomalies  must  already  be  known  and  flagged 
for  reaequisition.  In  the  future,  the  system  will  be  equipped  with  GPS  and/or  other 
positioning  systems  and  be  able  to  operate  in  a  detection  mode. 
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Figure  5-4  -  Sketch  of  the  TEMTADS  MP  2x2  sensor 
array  showing  the  position  of  the  four  sensors.  The 
standard  MR-20060 1  sensors  are  shown  schematically. 


Figure  5-5  -  The  NRL  TEMTADS  2x2  Man-Portable  Cart 

5.3,5  TEMTADS  MP  2x2  Cart  w/  Tri-axial  Receiver  Cubes 

The  upgraded  MP  system  with  the  tri-axial  reeeiver  eubes  is  eomprised  of  four  individual 
EMI  sensors  in  the  same  configuration  as  the  original  2x2  array,  as  shown  schematically 
in  Figure  5-6.  The  center-to-center  distance  is  40  cm  yielding  an  80  cm  x  80  cm  array. 


EM  Sensor 


Figure  5-6  -  Sketch  of  the  EMI  sensor  array  showing 
the  position  of  the  four  sensors.  The  tri-axial,  revised 
EMI  sensors  are  shown  schematically. 
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5.4  CALIBRATION  ACTIVITIES 


5.4.1  TEMTADS  Sensor  Calibration 

For  the  TEMTADS  family  of  sensors,  a  significant  amount  of  data  has  been  previously 
collected,  both  on  test  stands  and  under  field  conditions  at  our  test  field  [9]  and  during 
our  recent  demonstrations  at  APG  [2,10],  SLO  [11],  Bridgeport,  CT  [2],  and  at  the  former 
Camp  Butner,  NC  [12],  These  data  and  the  corresponding  fit  parameters  provide  us  with 
a  set  of  reference  parameters  including  those  of  clear  background  (i.e.  no  anomaly 
present). 

Daily  calibration  efforts  consisted  of  collecting  background  (no  anomaly)  data  sets 
periodically  throughout  the  day  and  during  the  APG  demonstrations.  The  background 
(no  anomaly)  data  sets  were  collected  at  known  quiet  spots  to  monitor  the  system  noise 
floor  and  for  background  subtraction  of  signal  data. 

5.4.2  Background  Data 

A  group  of  anomaly-free  areas  throughout  each  demonstration  site  were  identified  in 
advance  from  available  data,  MTADS  magnetometer  data  in  the  case  of  APG,  for 
example.  For  the  MP  system,  the  background  variation  is  presented  as  the  mean  and 
standard  deviation  of  the  four  monostatic  measured  signals  at  a  decay  time  of  42  ps  (7* 
time  gate).  For  the  APG  demonstration,  the  results  for  all  86  background  measurements 
taken  for  the  duration  of  the  demonstration  (August  30  -  September  2,  2010)  are  shown 
in  Figure  5-7.  Julian  date  codes  (day  of  the  year)  are  used  to  label  the  horizontal  axis. 
Table  5-1  provides  the  intraday  variations  of  the  mean  and  standard  deviation  quantities 
of  Figure  5-7. 

These  variations  have  been  correlated  in  the  field  with  both  ambient  temperature  and  the 
moisture  level  in  the  soil  surface  /  vegetation.  Background  levels  tend  to  be  high  in  the 
morning,  and  on  a  typical  field  day,  the  mornings  are  cool  and  dew  /  frost  may  be  present 
on  the  ground.  As  seen  in  Figure  5-7  on  Julian  dates  243  and  244  and  in  Reference  12,  as 
the  day  progresses  the  background  level  tends  to  decrease,  which  correlates  with 
increased  ambient  temperature  as  well  as  evaporation  of  any  moisture.  It  is  possible  that 
this  effect  is  caused  by  changes  in  the  coil  impedances  associated  with  changing 
temperature  and  /  or  humidity.  However,  we  cannot  rule  out  soil  /  vegetation  conductivity 
effects  on  the  background  signal.  Moisture  alone  can  cause  an  increased  background 
value,  as  was  seen  in  Reference  12  on  July  17,  2010.  During  rain  events,  the  background 
level  could  double  rapidly  and  would  recover  on  the  hour  time  scale. 


23 


Figure  5-7  -  Intra-  and  inter-  daily  variations  in  the  response  of  the  MP  system 
to  background  anomaly-free  areas  at  a  time  gate  of  42  ps  through  the  duration 
of  the  demonstration  at  APG. 

Table  5-1  -  Summary  of  the  daily  variation  in  the  mean  and  standard  deviation  of  the 
responses  measured  by  the  MP  system  for  the  background  areas  at  a  time  gate  of  42  ps 
at  APG. 


Date 

#  of  Bkgs. 

Mean  (mV/Amp) 

Std.  Dev. 
(mV/Amp) 

8/30/2010 

31 

23.74 

3.84 

8/31/2010 

26 

26.56 

3.64 

9/1/2010 

26 

29.49 

3.92 

9/2/2010 

3 

28.21 

4.62 

The  background  variation  analysis  results  for  the  Remington  Woods,  CT  (24 
measurements)  and  Dalecarlia  Woods,  DC  (13  measurements)  demonstrations  are  given 
in  Figure  5-8  and  Table  5-2  and  Figure  5-9  and  Table  5-3,  respectively.  The  Remington 
Woods,  CT  and  Dalecarlia  Woods,  DC  demonstrations  covered  the  periods  of  August  4  - 
6,  2009  and  May  21,  2010,  respectively. 
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Figure  5-8  -  Intra-  and  inter-  daily  variations  in  the  response  of  the 
TEMTADS  MP  2x2  array  to  background  anomaly-free  areas  at  a  time 
gate  of  42  ps  for  the  Remington  Woods,  CT  demonstration. 

Table  5-2  -  Summary  of  the  daily  variation  in  the  mean  and  standard  deviation  of  the 
responses  measured  by  the  TEMTADS  MP  2x2  array  for  the  background  areas  at  a 
time  gate  of  42  ps  at  the  Remington  Woods,  CT  demonstration  site. 


Date 

#  of  Bkgs. 

Mean  (mV/Amp) 

Std.  Dev. 
(mV/Amp) 

8/4/2009 

11 

22.39 

6.45 

8/5/2009 

7 

26.05 

5.38 

8/6/2009 

6 

23.25 

5.58 
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Figure  5-9  -  Intra-  and  inter-  daily  variations  in  the  response  of  the 
TEMTADS  MP  2x2  array  to  background  anomaly-free  areas  at  a  time  gate 
of  42  ps  for  the  Dalecarlia  Woods,  DC  demonstration. 

Table  5-3  -  Summary  of  the  daily  variation  in  the  mean  and  standard  deviation  of  the 
responses  measured  by  the  TEMTADS  MP  2x2  array  for  the  background  areas  at  a  time 
gate  of  42  ps  at  the  Dalecarlia  Woods,  DC  demonstration  site. 


Date 

#  of  Bkgs. 

Mean  (mV/Amp) 

Std.  Dev. 
(mV/Amp) 

5/21/2010 

13 

20.57 

5.55 

The  electrical  behavior  of  the  HH  sensor  is  somewhat  modified  from  that  of  the  standard 
TEMTADS  coil  due  to  the  geometry  of  its  construction,  as  discussed  in  Section  2.1.2. 
These  differences  require  that  a  different  decay  time,  118  ps,  be  used  for  monitoring  the 
background  response  of  the  system.  A  measurement  of  the  in-air  and  on-ground 
background  is  made  prior  to  and  after  each  measurement  cycle,  as  described  in  Section 
5.1.1.  All  808  measurement  of  in-air  and  on-ground  background  are  shown  in  Figure 
5-10.  Table  5-4  provides  the  intraday  variations  of  the  mean  and  standard  deviation 
quantities  of  Figure  5-10.  As  the  early-time  waveforms  of  the  two  sensors  differ  and 
different  time  gates  are  monitored,  the  magnitudes  of  the  background  responses  should 
not  be  directly  compared.  However,  the  relative  trends  can  be  compared. 
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Figure  5-10  -  Intra-  and  inter-  daily  variations  in  the  response  of  the  TEMTADS  Fland- 
Held  Sensor  to  baekground  anomaly-free  areas  at  a  time  gate  of  1 18  ps  for  the  APG 
demonstration. 


Table  5-4  -  Summary  of  the  daily  variation  in  the  mean  and  standard  deviation  of  the 
response  measured  by  the  TEMTADS  Fland-FIeld  Sensor  for  the  baekground  areas  at  a  time 
gate  of  1 1 8  ps  at  APG. 


Date 

#of 

Bkgs, 

Air  Mean 
(mV/Amp) 

Air  Std, 
Dev, 

(mV/Amp) 

Ground 

Mean 

(mV/Amp) 

Ground 
Std,  Dev, 
(mV/Amp) 

10/18/2010 

62 

0.71 

0.50 

0.97 

1.61 

10/19/2010 

61 

-0.27 

2.58 

0.13 

1.02 

10/20/2010 

91 

0.07 

0.53 

0.05 

4.27 

10/21/2010 

97 

0.36 

0.61 

0.64 

0.73 

10/22/2010 

42 

-0.42 

2.14 

0.11 

0.22 

10/25/2010 

51 

0.65 

0.66 

1.40 

0.91 

5,4,3  Performance  at  APG  -  60mm  Mortars 

For  recent  live  site  demonstrations,  the  day-to-day  performance  of  a  technology  is  often 
demonstrated  through  the  use  of  an  Instrument  Verification  Strip  (I VS).  The  intent  of  an 
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IVS  is  to  provide  the  ability  to  verify  the  repeatability  of  the  system  response  on  several 
examples  of  items  of  interest.  Each  emplaced  item  in  the  IVS  would  be  measured  twice 
daily,  once  before  starting  the  data  collection  process  and  a  second  time  before  shutting 
the  system  down  at  the  end  of  each  day.  The  APG  Standardized  UXO  Test  Site  has  a 
previously  emplaced,  large  (66  item)  Calibration  Area  for  demonstrators  to  use  and  a 
single,  shallow  pit  for  placing  other  objects.  As  such,  demonstrations  at  APG  measure 
the  Calibration  Area  items  a  single  time  prior  to  moving  on  to  the  Blind  Grid  and  Open 
Field  Areas.  Therefore  to  demonstrate  the  day-to-day  variability  of  the  recovered 
parameters  for  each  of  the  sensor  technologies,  the  results  for  a  single  munitions  type  are 
shown  in  aggregate  for  each  system.  Except  for  the  Calibration  Area,  the  ground  truth  is 
held  close  at  ATC  and  not  available  to  the  demonstrators.  Items  believed  to  be  60mm 
mortars  are  used  in  the  following  examples.  No  IVS-like  facilities  were  available  at 
Remington  Woods,  CT  or  Dalecarlia  Woods,  DC,  so  no  such  comparisons  are  shown. 

For  reference,  the  performance  of  the  TEMTADS  5x5  array  is  shown  in  Figure  5-11. 

The  fit-result  principle  magnetic  polarizabilities  are  shown  in  black,  red,  and  green, 
respectively.  The  mean  and  a  2g  envelope  for  the  axial  and  transverse  polarizabilities  are 
shown  in  magenta  and  black,  respectively. 
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Figure  5-1 1  -  TEMTADS  5x5  array  derived  response  coefficients  for  all  items  at 
APG  classified  as  60mm  mortars. 

The  analysis  results  for  the  same  items  with  the  MP  system  and  HH  sensor  are  shown  in 
Figure  5-12  and  Figure  5-13,  respectively.  The  HH  system’s  performance  was 
quantitatively  similar  to  that  of  the  full  TEMTADS  5x5  array.  The  performance  of  the 
MP  system  was  significantly  degraded.  See  Section  7.8  for  further  discussion  of  the  MP 
system  performance. 
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Figure  5-12  -  TEMTADS  MP  2x2  Cart  array  derived  response  eoeffieients  for  all 
items  at  APG  elassified  as  60mm  mortars. 


10.0 


0.10 


CO 

< 

CD 

CD 

C 

o 

Q. 

CD 

CD 

cc 

c 

g 

ro 

N 


0.01 
o 
Q. 


0.001 

0.1 


1.0 

Time  Gate  (msecs) 


10.0 


EM61Mk2,  4 


pi  -  black 
p2  -  red 
p3  -  green 
fix  ±20  -  magenta 
pLt±20  -  blue 


Figure  5-13  -  TEMTADS  Fland-FIeld  Sensor  derived  response  eoeffieients  for  all 
items  at  APG  elassified  as  60mm  mortars. 

5.5  DATA  COLLECTION  PROCEDURES 

5,5,1  Scale  of  the  Demonstrations 

The  HH  sensor  demonstration  was  conducted  at  the  APG  Standardized  UXO  Test  Site. 
The  Calibration  Area  and  the  Blind  Grid  Areas  were  surveyed.  Only  those  cells  in  the 
Blind  Grid  Area  that  were  on  the  union  of  the  TEMTADS  (MR-200601)  and  SAINT 
(MR-200810)  target  lists  were  surveyed  with  the  HH  sensor.  The  MP  system 
demonstration  at  the  same  site  covered  the  Calibration  Area,  and  the  Blind  Grid  and 
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Indirect  Fire  Areas,  using  the  original  TEMTADS  target  list.  The  Remington  Woods  and 
Dalecarlia  Woods  demonstration  were  conducted  on  the  respective  sites  using  provided 
target  lists  from  the  ongoing  remediation  efforts.  For  all  sites,  the  locations  on  the  target 
lists  were  previously  reacquired  and  flagged. 

5.5.2  Sample  Density 

The  EMI  data  spacing  for  the  MP  system  is  fixed  at  40  cm  in  both  directions  by  the  array 
design.  Two  set  of  data  were  collected  for  each  flag  position  as  described  in  Section 
5.1.2.  The  HH  sensor  data  are  collected  on  a  6x6  grid  template  with  15-cm  grid  spacing. 
In-air  and  ground  background  measurements  are  taken  on  a  known  quiet  spot  within  a 
few  steps  of  the  flag  location. 

5.5.3  Quality  Checks 

Preventative  maintenance  inspections  were  conducted  at  least  once  a  day  by  all  team 
members,  focusing  particularly  on  the  sensors  and  cabling.  Any  deficiencies  were 
addressed  according  to  the  severity  of  the  deficiency.  Parts,  tools,  and  materials  for  many 
maintenance  scenarios  were  available  in  the  system  spares  inventory  which  was  on  site. 

Two  data  quality  checks  were  performed  on  the  EMI  data.  After  background  subtraction, 
the  data  are  plotted  as  a  function  of  time  for  each  transmitter/receiver  pair.  An  example 
plot  is  shown  in  Eigure  5-14  for  the  MP  system  and  APG  Calibration  Area  item  G02,  a 
37mm  projectile  buried  at  a  depth  of  24  cm  below  the  surface.  The  plots  were  visually 
inspected  to  verify  that  there  was  a  well-defined  anomaly  without  extraneous  signals  or 
dropouts.  Eurther  QC  evaluation  on  the  transmit/receive  cross  terms  was  based  on  the 
dipole  inversion  results.  An  example  of  the  inversion  results  (principle  polarizability 
decays)  is  shown  in  Eigure  5-15  for  the  data  shown  in  Eigure  5-14.  Our  experience  has 
been  that  data  glitches  show  up  as  a  degraded  match  of  the  extracted  response 
coefficients  to  the  reference  values,  when  appropriate.  This  is  quantitatively  seen  as  a 
reduced  fit  coherence.  The  fit  coherence  is  a  value  (0-1)  reflecting  how  well  the  fit 
result  response  coefficients  reproduce  the  collected  data.  Qualitative  evaluation  is  also 
conducted  by  visual  inspection  of  several  QC  plots  by  the  data  analyst. 
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Figure  5-14  -  TEMTADS  MP  2x2  Cart  QC  plot  for  APG  Calibration  Area  item  G002,  a  37mm 
projectile  at  a  depth  of  24  cm  below  the  surface. 


Time  (ms) 

Figure  5-15  -  TEMTADS  MP  2x2  Cart  derived  response  coefficients 
for  APG  Calibration  Area  item  G002,  a  37mm  projectile  at  a  depth  of 
24  cm  below  the  surface.  The  blue  lines  are  the  fit  results  for  the 
collected  data  and  the  red  lines  indicate  a  library  entry  for  a  37mm 
projectile. 

Any  data  set  deemed  unsatisfaetory  by  the  data  analyst  was  flagged  and  not  proeessed 
further.  The  anomaly  eorresponding  to  the  flagged  data  was  logged  for  re-aequisition  by 
the  field  team. 

5,5,4  Data  Handling 

Data  were  stored  electronieally  on  the  backpack  data  acquisition  computer  hard  drive. 
Approximately  every  two  hours,  the  field  data  were  copied  onto  removable  media  and 
transferred  to  the  onsite  data  analyst  for  QC/analysis.  The  data  were  moved  onto  the  data 
analyst’s  computer  and  the  media  was  recycled.  Raw  data  and  analysis  results  were 
backed  up  from  the  data  analyst’s  computer  to  external  hard  disks  daily.  These  results 
are  archived  on  an  internal  file  server  at  SAIC  at  the  end  of  the  survey.  All  field  notes  / 
activity  logs  were  written  in  ink  and  stored  in  archival  laboratory  notebooks.  These 
notebooks  are  archived  at  NRL  and  SAIC.  Dr.  Tom  Bell  is  the  POC  for  obtaining  data 
and  other  information.  His  contact  information  is  provided  in  Appendix  A  of  this  report. 

5,6  VALIDATION 

Validation  of  the  performance  of  these  technologies  comes  primarily  from  comparison  of 
the  classification  results  of  the  data  analysis  to  the  ground  truth.  In  the  case  of  the  APG 
Standardized  UXO  Test  Site,  the  ground  truth  is  known  to  the  site  managers  and  no 
intrusive  investigation  is  required.  For  the  Remington  Woods  and  Dalecarlia  Woods 
sites,  the  targets  selected  for  investigation  were  already  scheduled  for  intrusive 
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investigation  as  part  of  the  ongoing  cleanup  efforts  at  each  site.  Ground  truth  results 
were  provided  after  the  intrusive  investigations  were  complete. 

5.6.1  Aberdeen  Proving  Ground,  MD 

With  the  exception  of  the  Calibration  Areas,  the  ground  truth  for  the  Standardized  sites  is 
held  back  from  individual  technology  demonstrators  to  preserve  the  utility  of  the  Blind 
Grid  and  the  Open  Field  Areas.  Results  from  the  Blind  Grid  and  the  Indirect  Fire  Area 
(for  the  MP  system)  were  submitted  to  ATC  for  performance  evaluation.  Scoring  results 
have  been  received  and  are  available  [13,14].  A  summary  of  the  results  are  given  in 
Section  7.0. 

5.6.2  Remington  Woods,  CT 

The  two  demonstrations  at  the  Remington  Woods,  CT  conducted  for  a  side-by-side 
comparison  with  the  current  state-of-the-art  UXO  classification  techniques  being  used  on 
site.  Ongoing  UXO  remediation  is  based  on  the  results  of  an  EM61-HH  cued-template 
survey  following  an  EM61-Mk2  detection  survey.  The  list  of  targets  investigated  with 
the  MP  system  was  a  subset  of  the  list  prioritized  for  intrusive  investigation  based  on  the 
EM61  surveys.  Each  Eall,  the  items  indicated  for  investigation  on  the  prioritized  EM61- 
based  dig  list  are  excavated.  Ground  truth  and  photography  are  then  available  for  the 
targets  investigated  by  the  MP  system.  A  summary  of  the  results  is  given  in  Section  7.0. 

5.6.3  Spring  Valley,  Washington,  DC 

The  demonstration  at  the  Dalecarlia  Woods,  DC  was  conducted  as  a  side-by-side 
comparison  with  the  target  prioritization  techniques  being  used  on  site.  Ongoing  UXO 
remediation  is  based  on  the  results  of  EM61-Mk2  and  magnetometer  surveys.  The  list  of 
targets  investigated  with  the  MP  system  was  a  subset  of  the  list  prioritized  for  intrusive 
investigation.  The  items  indicated  for  intrusive  investigation  on  the  prioritized  dig  list 
were  excavated.  Ground  truth  was  provided  for  the  targets  investigated  by  the  MP 
system.  A  summary  of  the  results  is  given  in  Section  7.0. 

6.0  DATA  ANALYSIS  PLAN 

6.1  PREPROCESSING 

6.1.1  TEMTADS  Hand-Held  Sensor 

The  HH  sensor  has  one  EMI  sensor  with  concentric  transmitter  and  receiver  coils.  Eor 
each  transmit  pulse,  we  record  the  transient  decay  response  at  the  receiver  (12,500 
points).  The  recorded  data  are  then  binned  into  a  series  of  time  gates  for  improved 
manageability  and  increased  signal-to-noise.  Normally  we  use  122  logarithmically 
spaced  time  gates.  In  preprocessing,  the  recorded  signals  are  normalized  by  the 
transmitter  currents  to  account  for  any  transmitter  variations.  On  average  the  peak 
transmitter  current  is  approximately  7.5  Amps.  Due  to  a  mis-calibration  in  the  data 
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acquisition  software,  the  reported  eurrents  were  half  the  aetual,  approximately  3.75  A. 
This  issue  has  sinee  been  resolved.  Deeay  time  is  measured  from  the  time  that 
transmitter  turn-off  is  initiated.  We  subtraet  0.028  ms  from  the  nominal  gate  times  to 
aeeount  for  time  delay  due  to  effects  of  the  reeeive  coil,  electronics,  and  the  Tx  turn-off 
delay  [15].  The  eorreetion  was  determined  empirieally  by  eomparing  measured 
responses  for  test  spheres  with  theory.  Measured  responses  inelude  interfering  signals 
due  to  transmitter  ringing  and  related  artifaets  out  to  about  0.160  msee.  Consequently  we 
only  inelude  response  beyond  1 1 8  ps  in  our  analysis  as  the  baekground  is  too  large  and 
varying  to  be  reliably  subtraeted  at  earlier  times.  This  leaves  99  gates  spaeed 
logarithmieally  between  0.1 18  ms  and  25.35  ms. 

The  background  response  is  subtraeted  from  eaeh  target  measurement  using  data 
colleeted  in  a  nearby  target-free  region  measured  at  the  same  height  as  the  template.  All 
baekground  measurements  were  inter-eompared  to  evaluate  baekground  variability  and 
identify  outliers  whieh  may  eorrespond  to  measurements  over  non-ferrous  targets.  In 
previous  testing  at  our  Blossom  Point  test  field  and  during  other  demonstrations, 
significant  background  variation  was  not  observed.  It  has  been  possible  to  use  blank 
ground  measurements  from  100  meters  away  for  baekground  subtraetion.  Changes  in 
moisture  eontent  and  outside  temperature  have  been  shown  to  eause  variation  in  the 
baekgrounds,  neeessitating  eare  when  eollecting  data  after  weather  events  such  as  rain. 

6.1.2  TEMTADS  MP  2x2  Cart 

The  MP  system  has  four  sensor  elements,  eaeh  eomprised  of  a  transmitter  eoil  and  a 
vertieally-oriented  reeeiver  eoil.  For  eaeh  transmit  pulse,  the  responses  at  all  of  the 
reeeivers  are  recorded.  This  results  in  16  possible  transmitter  /  reeeiver  eombinations  in 
the  data  set  (4  transmitters  x  4  reeeiver  cubes).  In  preproeessing,  the  reeorded  signals  are 
normalized  by  the  peak  transmitter  eurrent  to  aeeount  for  any  variation  in  the  transmitter 
output.  On  average,  the  peak  transmitter  eurrent  is  approximately  7.5  Amps.  Due  to  a 
mis-ealibration  in  the  data  aequisition  software,  the  reported  currents  were  half  the  aetual, 
3.75  A.  This  issue  has  sinee  been  resolved.  Although  the  data  aequisition  system  reeords 
the  signal  over  122  logarithmieally-spaeed  time  gates,  the  measured  responses  over  the 
first  7  gates  inelude  interfering  signals  due  to  transmitter  ringing  and  related  artifaets  and 
are  disearded.  We  subtract  0.028  ms  from  the  nominal  gate  times  to  aeeount  for  time 
delay  due  to  effeets  of  the  reeeive  eoil,  eleetronies,  and  the  Tx  turn-off  delay  [15].  The 
delay  was  determined  empirieally  by  eomparing  measured  responses  for  test  spheres  with 
theory.  This  leaves  115  gates  spaced  logarithmieally  between  0.042  ms  and  25.35  ms. 

The  background  response  is  subtraeted  from  eaeh  target  measurement  using  data 
eolleeted  at  a  nearby  target-free  baekground  loeation.  As  few  measurement  eyeles  are 
required  for  the  MP  system  (8  vs.  40);  the  MP  system  ean  eolleet  data  over  more 
targets/hour  than  the  HH  sensor  for  a  given  set  of  data  acquisition  parameters.  Based  on 
previous  experienee  with  the  MP  system  and  the  TEMTADS  5x5  array,  a  baekground 
measurement  for  the  MP  system  was  made  approximately  every  30  minutes.  The  same 
eaveats  mentioned  in  the  previous  Seetion  apply. 
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6.2  TARGET  SELECTION  FOR  DETECTION 


6.2.1  Aberdeen  Proving  Ground,  MD 

The  anomaly  list  for  the  Blind  Grid  and  the  Indirect  Fire  Areas  were  the  same  ones  as 
used  for  the  TEMTADS  5x5  array  demonstration  in  June  2008  [7]. 

6.2.2  Remington  Woods,  CT 

DuPont  Corp.  and  URS  Corp.  are  currently  involved  in  an  ongoing  UXO  remediation 
effort  at  this  site.  The  initial  target  detection  is  based  on  the  results  of  an  EM61-Mk2 
survey.  After  analysis  and  prioritization,  a  cued-template  EM61-HH  survey  is  conducted 
to  further  refine  the  diglist  prioritization.  A  prioritized  diglist  is  then  generated  with  three 
classes  of  targets.  Eikely  Munitions  (Class  1),  Possible  Munitions  (Class  2),  and  Eikely 
Clutter  (Class  3).  All  Class  1  targets  are  excavated  using  robotically-controlled 
excavators  and  blast  shield.  All  Class  2  targets  are  investigated  by  UXO  technicians 
manually.  A  statistical  sampling  (-'10%)  of  the  Class  3  targets  are  investigated  for 
quality  control  purposes.  The  list  of  targets  investigated  with  the  MP  system  was  a  subset 
of  the  original  prioritized  dig  list  including  the  Class  1,  Class  2,  and  the  sampled  Class  3 
targets. 


6.2.3  Spring  Valley,  Washington,  DC 

The  USAGE,  Baltimore  District  has  an  established,  ongoing  remediation  project  at  the 
Spring  Valley  EUDS.  Based  on  extensive  geophysical  data  and  review  by  the  Anomaly 
Review  Board,  dig  lists  are  prepared  for  intrusive  investigation.  A  small  segment  of  the 
dig  list  for  2010  was  selected  for  investigation  based  on  schedule.  All  items  on  the 
segment  of  the  dig  list  were  investigated  and  ground  truth  was  provided. 

6.3  PARAMETER  ESTIMATION 

The  raw  signature  data  from  TEMTADS  sensors  reflect  details  of  the  sensor/target 
geometry  as  well  as  inherent  EMI  response  characteristics  of  the  targets  themselves.  In 
order  to  separate  out  the  intrinsic  target  response  properties  from  sensor/target  geometry 
effects,  we  invert  the  signature  data  to  estimate  principal  axis  magnetic  polarizabilities 
for  the  targets.  The  TEMTADS  data  are  inverted  using  the  standard  induced  dipole 
response  model  wherein  the  effect  of  eddy  currents  set  up  in  the  target  by  the  primary 
field  is  represented  by  a  set  of  three  orthogonal  magnetic  dipoles  at  the  target  location 
[16].  The  measured  signal  is  a  linear  function  of  the  induced  dipole  moment  m,  which 
can  be  expressed  in  terms  of  a  time  dependent  polarizability  tensor  B  as 

m  =  UBU^Ho 

where  U  is  the  transformation  matrix  between  the  physical  coordinate  directions  and  the 
principal  axes  of  the  target  and  Ho  is  the  primary  field  strength  at  the  target.  The 
eigenvalues  Pi(t)  of  the  polarizability  tensor  are  the  principal  axis  polarizabilities. 
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Given  a  set  of  measurements  of  the  target  response  with  varying  geometries  or  "look 
angles"  at  the  target,  the  data  ean  be  inverted  to  determine  the  loeal  (X,Y,Z)  location  of 
the  target,  the  orientation  of  its  principal  axes  ((t),0,\)/),  and  the  principal  axis 
polarizabilities  (Pi,P2,P3).  The  basic  idea  is  to  search  out  the  set  of  nine  parameters 
(X,Y,Z,(t),0,\|/,Pi,P2,P3)  that  minimizes  the  difference  between  the  measured  responses 
and  those  calculated  using  the  dipole  response  model.  Since  the  system  currently  does 
not  know  or  record  the  location  or  orientation  of  the  cart,  target  location  and  orientation 
are  known  well  locally  but  not  well  geo-referenced. 

For  TEMTADS  data,  inversion  is  accomplished  by  a  two-stage  method.  In  the  first  stage, 
the  target’s  (X,Y,Z)  dipole  location  is  solved  for  non-linearly.  At  each  iteration  within 
this  inversion,  the  nine  element  polarizability  tensor  (B)  is  solved  linearly.  We  require 
that  this  tensor  be  symmetric;  therefore,  only  six  elements  are  unique.  Initial  guesses  for 
X  and  Y  are  determined  by  a  signal-weighted  mean.  The  routine  normally  loops  over  a 
number  of  initial  guesses  in  Z,  keeping  the  result  giving  the  best  fit  as  measured  by  the 
chi-squared  value.  The  non-linear  inversion  is  done  simultaneously  over  all  time  gates, 
such  that  the  dipole  (X,Y,Z)  location  applies  to  all  decay  times.  At  each  time  gate,  the 
eigenvalues  and  angles  are  extracted  from  the  polarizability  tensor. 

In  the  second  stage,  six  parameters  are  used;  the  three  spatial  parameters  (X,Y,Z)  and 
three  angles  representing  the  yaw,  pitch,  and  roll  of  the  target  (Euler  angles  (t),0,\|/).  Here 
the  eigenvalues  of  the  polarizability  tensor  are  solved  for  linearly  within  the  6-parameter 
non-linear  inversion.  In  this  second  stage  both  the  target  location  and  its  orientation  are 
required  to  remain  constant  over  all  time  gates.  The  value  of  the  best  fit  X,Y,Z  from  the 
first  stage,  and  the  median  value  of  the  first-stage  angles  are  used  as  an  initial  guess  for 
this  stage.  Additional  loops  over  depth  and  angles  are  included  to  better  ensure  finding 
the  global  minimum. 

Eigure  6-1  shows  an  example  of  the  principal  axis  polarizabilities  determined  from 
TEMTADS  array  data.  The  target,  a  mortar  fragment,  is  a  slightly  bent  plate  about  0.5 
cm  thick,  25  cm  long,  and  15  cm  wide.  The  red  curve  is  the  polarizability  when  the 
primary  field  is  normal  to  the  surface  of  the  plate,  while  the  green  and  blue  curves 
correspond  to  cases  where  the  primary  field  is  aligned  along  each  of  the  edges. 

Not  every  target  on  the  target  list  exhibited  a  strong  enough  TEM  response  to  support 
extraction  of  target  polarizabilities.  All  of  the  data  were  run  through  the  inversion 
routines,  and  the  results  manually  screened  to  identify  those  targets  that  could  not  be 
reliably  parameterized.  Several  criteria  were  used:  signal  strength  relative  to  background, 
dipole  fit  error  (difference  between  data  and  model  fit  to  data),  and  the  visual  appearance 
of  the  polarizability  curves. 
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Time  (msec) 

Figure  6-1  -  Principal  axis  polarizabilities  for  a  0.5  cm  thick 
by  25  cm  long  by  1 5  cm  wide  mortar  fragment. 

6.4  CLASSIFICATION 

Target  classification  is  based  on  a  library  matching  procedure  wherein  we  compare  the 
quality  of  both  an  unconstrained  dipole  inversion  of  the  TEM  array  data  and  the  ratio,  p. 
p  is  defined  as  the  ratio  of  the  quality  of  an  unconstrained  dipole  fit  of  the  TEM  data  to 
the  quality  of  a  dipole  fit  constrained  by  principal  axis  polarizabilities  drawn  from  the 
signature  library.  Eit  quality  is  the  squared  correlation  coefficient  between  the  model  fit 
and  the  data.  If  p  is  equal  to  one,  then  the  library  item  is  as  good  a  match  to  the  data  as 
possible.  If  the  value  of  p  is  small,  then  the  library  item  is  a  poor  match.  Eor  the 
unconstrained  inversion,  we  utilize  an  algorithm  which  compares  our  derived 
polarizabilities  with  a  library  of  known  target  signatures.  The  match  is  based  on  three 
criteria:  the  amplitude  of  the  primary  polarizability,  and  the  ratio  of  the  second  and  third 
polarizabilities  to  the  first.  We  have  computed  match  metrics,  each  of  which  runs  from  0 
(terrible  match)  to  1  (perfect  match). 

6.5  TRAINING 

Our  experience  with  these  sensors  has  been  that  principle  polarizabilities  determined 
from  in-air  measurements  are  indistinguishable  Ifom  those  determined  from 
measurements  taken  over  buried  targets.  We  have  an  extensive  collection  of  inert 
military  munitions  collected  from  many  sources  which  were  measured  at  our  home 
facility  using  the  TEMTADS  family  of  sensors  mounted  on  a  test  stand.  We  have  also 
assembled  a  fairly  extensive  polarizability  database  for  clutter  items  recovered  from 
several  different  sites.  This  was  used  as  training  data  for  establishing  UXO/clutter 
discrimination  boundaries  on  the  coherence  ratio  p  and  on  the  direct  comparison  metric. 
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6,5.1  Aberdeen  Proving  Ground 


We  collected  training  data  in  air  for  all  of  the  14  standard  APG  ordnance  targets  with 
these  EMI  sensors.  These  data  were  used  for  the  fit  library  entries.  Many  of  the  targets 
are  composites  of  two  or  more  distinct  parts,  like  a  steel  body  combined  with  an 
aluminum  tail  assembly.  Depending  on  the  distance  between  the  sensors  and  the  target, 
such  items  can  exhibit  a  range  of  slightly  different  EMI  signatures  corresponding  to 
excitation  from  different  directions.  We  include  measurements  with  the  target  oriented 
nose  up,  towards  the  sensor  array,  nose  down,  away  from  the  array,  fiat  and  obliquely. 

6.5.2  Remington  Woods,  CT 

Several  recovered  munitions  were  available  on  site  for  measurement  after  being  certified 
as  inert.  These  items  along  with  a  set  of  munitions  surrogates  developed  by  DuPont  and 
URS  were  measured  in-air  on  site  during  the  demonstrations. 

6.5.3  Dalecarlia  Woods,  DC 

Training  items  unique  to  this  site  were  not  readily  available  and  our  existing  inventory  of 
library  signatures  was  used  for  this  demonstration. 

6.6  DATA  PRODUCT  SPECIFICATIONS 

6,6,1  Aberdeen  Proving  Ground 

Eor  the  demonstrations  conducted  at  the  APG  Standardized  UXO  Test  Site,  we  used  the 
standard  reporting  templates  for  the  Blind  Grid  and  the  Open  Eield  shown  below  in 
Eigure  6-2  and  Eigure  6-3.  The  metrics  in  Section  3.0  were  calculated  directly  from  the 
Scoring  Report  provided  by  the  Standardized  Test  Site  administrators.  The 
Discrimination  Stage  value  was  the  highest  coherence  ratio  for  a  UXO  library  entry 
(using  only  60mm  and  81mm  mortars  and  the  105mm  projectile  for  the  Indirect  Eire 
Area).  Classification  and  Type  were  determined  from  the  library  matching  procedure. 
Depth  and  Dip  values  come  from  the  dipole  inversion  results,  the  other  location  and 
orientation  values  are  not  well  georeferenced  for  these  systems. 
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Figure  6-2  -  Reporting  Template  for  APG  Blind  Grid. 


Figure  6-3  —  Reporting  Template  for  APG  Indirect  Fire  Area. 

6,6.2  Remington  Woods,  CT 

The  ongoing  efforts  at  the  Remington  Woods  site  provide  a  starting  point  for  the  MP 
system  dig  list.  The  format  of  the  ranked  dig  list  is  given  in  Figure  6-4.  The  first  several 
columns  provide  the  Target  ID  #  and  the  ranking  from  the  EM61-MK2  and  EM-61  HH 
surveys  for  each  target  to  be  investigated.  The  “Eield  Comments”  column  is  provided  to 
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capture  any  notes  from  the  field  team,  e.g.  “surface  item  present.”  The  ranking  based  on 
the  MP  system  results  are  given  next  in  the  “TEM  Category”  eolurnn,  using  the  same 
ranking  system  as  was  used  for  the  EM61  surveys.  As  part  of  the  ongoing  efforts  onsite, 
a  series  of  munitions  simulants  were  machined  for  calibration  purposes.  Our  results  have 
shown  that  the  simulants  have  their  own  unique  decay  signatures  as  compared  to  the 
munitions  they  were  designed  to  simulate.  If  the  fit  results  for  a  target  were  a  good  mateh 
to  a  simulant  signature,  it  was  noted  in  the  “Simulant  Match”  column.  The  remaining 
two  ground-truth  columns  were  populated  after  dig  list  submission  and  intrusive 
investigation  were  eomplete. 


Figure  6-4  —  Reporting  Template  for  Remington  Woods,  CT. 

6,6.3  Dalecarlia  Woods,  DC 

The  prioritized  diglist  for  the  Dalecarlia  Woods  demonstration  delineated  the  targets  into 
four  categories,  1— Likely  Clutter,  2— Cannot  Deeide,  3— Likely  UXO,  4-Cannot  Analyze. 
Overall  ranking  for  target  signatures  that  could  be  analyzed  started  with  the  most  likely  to 
be  clutter  (eategory  #1)  and  increased  through  categories  2  &  3.  Those  signatures  that 
were  categorized  as  “Cannot  Analyze”  were  appended  to  the  end  of  the  diglist.  The 
format  of  the  ranked  dig  list  is  given  in  Eigure  6-5. 
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Target  ID  # 

Rank 

Category 

Type 

Depth  (m) 

1 

2 

3 

4 

5 

6 

7 

Figure  6-5  —  Reporting  Template  for  the  Daleearlia  Woods,  DC  demonstration. 

7.0  PERFORMANCE  ASSESSMENT 

The  performance  objectives  for  the  APG  demonstrations  are  summarized  in  Table  3-1 
and  are  repeated  here  as  Table  7-1.  The  results  for  each  criterion  are  subsequently 
discussed  in  the  following  sections.  For  the  Remington  Woods  and  Spring  Valley 
demonstrations,  the  MP  system  was  invited  to  participate  in  ongoing  remediation  efforts 
without  formal  demonstration  plans.  The  performance  at  each  site  is  discussed  in 
Sections  7.6  and  7.7,  respectively. 

Performance  objectives  for  the  demonstrations  are  given  as  a  basis  for  the  evaluation  of 
the  performance  and  costs  of  the  demonstrated  technologies.  Since  these  are 
classification  technologies,  the  performance  objectives  focus  on  the  second  step  of  the 
UXO  remediation  problem;  that  of  target  classification  as  UXO,  clutter,  etc.  We  assume 
that  the  anomalies  from  all  targets  of  interest  have  been  detected  and  have  been  included 
on  the  target  list. 
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Table  7-1  -  Performance  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Success? 

(Yes/No) 

Quantitative  Performance  Objectives 

Correct 
classification 
of  targets  of 
interest 

Number  of 
targets  of 
interest 
identified 

•  Prioritized 
dig  list 

•  Scoring 
report  from 
APG 

95%  correct 
identification 
of  all  targets  of 
interest 

HH  -  Yes 
MP-No 

Reduction  of 
False  Alarms 

Number  of  false 

alarms 

eliminated 

•  Prioritized 
dig  list 

•  Scoring 
report  from 
APG 

Reduction  of 
false  alarms  by 
50%  or  more 
with  95% 

correct 

identification  of 
munitions 

HH  -  Yes 
MP-No 

Cued 

Production 

Rate 

Number  of  cued 
targets 

investigated  per 
day 

Log  of  field 
work 

HH  -  50/day 

MP  -  200/day 

HH  -  Yes 
MP  -  Yes 

Analysis 

Time 

Average  time 
required  for 
inversion  and 
classification 

Log  of 
analysis  work 

15  min/target 

HH  -  Yes 
MP  -  Yes 

Qualitative  Performance  Objective 

Ease  of  Use 

System  can  be 
used  in  the  field 
without 
significant 
issues 

Team 

feedback 

Lield  team  has 
no  significant 
issues  to  report 

HH  -  Yes 
MP  -  Yes 

Reliability 

and 

Robustness 

•  Number  of 
operational 
hours  recorded 
per  day 

•  Number  of 
significant 
technical  issues 

•  Lield  logs  of 
operational 
hours  per  day 

•  Lield  logs  of 
significant 
technical 
issues 

•  >  6  hour/day 

•  <  1  significant 
technical  issue 
per  day 

HH  -  Yes 
MP  -  Yes 
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7.1  CORRECT  CLASSIFICATION  AND  REDUCTION  OF  FALSE  ALARMS 

7.1.1  Correct  Classification  of  Targets  of  Interest 

This  is  one  of  the  two  primary  measures  of  the  elassifieation  value  of  the  data  colleeted 
by  these  sensor  systems.  By  eolleeting  high-quality,  preeisely  relatively-loeated  data,  it 
should  be  possible  to  diseriminate  munitions  from  serap  and  frag  with  some  effieiency. 
We  expeeted  to  properly  elassify  a  large  pereentage  of  the  seeded  munitions  items. 

7.1. 1.1.  Metric 

At  a  seeded  test  site  sueh  as  the  APG  Standardized  UXO  Test  Site,  the  metric  for 
classification  efficiency  is  straightforward.  We  prepared  a  ranked  dig  list  from  the  survey 
data  with  a  UXO  /  Clutter  decision  for  each  Blind  Grid  cell  and  for  each  location  in  the 
Indirect  Fire  Area  that  the  MP  system  investigated.  ATC  personnel  used  their  automated 
scoring  algorithms  to  assess  our  results. 

7.1. 1.2.  Data  Requirements 

The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators. 
Our  ranked  dig  lists  were  the  input  for  this  metric  and  ATC’s  standard  scoring  was  the 
output. 

7.1. 1.3.  Success  Criteria 

The  objective  was  considered  to  be  met  for  each  demonstration  if  more  than  95%  of  the 
seeded  munitions  items  were  correctly  classified. 

7.1.2  Objective:  Reduction  of  False  Alarms 

This  is  the  second  of  the  two  primary  measures  of  the  classification  value  of  the  data 
collected  by  these  technologies.  By  collecting  high-quality,  precisely  relatively-located 
data,  it  should  be  possible  to  discriminate  munitions  from  scrap  and  frag  with  some 
efficiency.  We  expected  to  properly  classify  a  large  percentage  of  the  clutter  as  such. 

7.1.2. 1.  Metric 

At  a  seeded  test  site  such  as  the  APG  Standardized  UXO  Test  Site,  the  metric  for  false 
alarm  elimination  is  straightforward.  We  prepared  a  ranked  dig  list  from  the  survey  data 
with  a  UXO  /  Clutter  decision  for  each  Blind  Grid  cell  and  for  each  location  in  the 
Indirect  Fire  Area  that  the  MP  system  investigated.  ATC  personnel  used  their  automated 
scoring  algorithms  to  assess  our  results. 
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IA.2.2.  Data  Requirements 


The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators. 
Our  ranked  dig  lists  were  the  input  for  this  metric  and  ATC’s  standard  scoring  was  the 
output. 


7.1,2,3.  Success  Criteria 

The  objective  was  considered  met  if  more  than  50%  of  the  non-munitions  items  were 
labeled  as  no-dig  while  retaining  95%  of  the  munitions  items  on  the  dig  list. 

7.1,3  Results 

These  Objectives  were  successfully  met  for  the  HH  sensor  and  partially  met  for  the  MP 
system.  The  HH  sensor  surveyed  anomalies  from  the  union  of  the  TEMTADS  and  the 
SAINT  target  lists  for  the  Blind  Grid  Area.  The  MP  system  surveyed  the  same  anomalies 
in  the  Blind  Grid  and  Indirect  Fire  Areas  surveyed  during  the  TEMTADS  5x5  array 
demonstration.  For  the  HH  sensor,  the  Blind  Grid  Area  discrimination  stage  results  are 
summarized  in  Table  7-2  and  Table  7-3  (subsets  of  Table  6a  of  Reference  13),  broken  out 
by  munitions  type  and  emplacement  depth.  For  the  MP  system,  the  Blind  Grid  Area 
discrimination  stage  results  are  summarized  in  Table  7-4  and  Table  7-5  (subsets  of  Table 
6a  of  Reference  14),  broken  out  by  munitions  type  and  emplacement  depth.  For  the  MP 
system,  the  Indirect  Fire  Test  Area  discrimination  stage  results  are  summarized  in  Table 
7-8  and  Table  7-9  (subsets  of  Table  6c  of  Reference  14),  broken  out  by  munitions  type 
and  emplacement  depth.  The  Discrimination  Stage  Probability  of  Detection  is 
defined  as  the  number  of  correctly  identified  munitions  divided  by  the  number  of 
emplaced  munitions,  and  the  corresponding  Probability  of  False  Positive  Pfp‘*‘*‘^  is  the 
number  of  clutter  items  incorrectly  identified  as  munitions  divided  by  the  number  of 
emplaced  clutter  items.  For  reference,  the  corresponding  TEMTADS  5x5  array  results 
are  provided  in  Table  7-6,  Table  7-7,  Table  7-10,  and  Table  7-11. 

The  HH  sensor  successfully  met  this  objective  with  96%  correct  identification  of  all 
targets  of  interest  in  the  Blind  Grid  Area.  The  weakest  performance  was  in  the  8-1 2D 
depth  category.  The  MP  system  came  very  close  to,  but  did  not  successfully  meet  these 
criteria  in  either  survey  area.  The  MP  system  performance  was  94%  for  the  Blind  Grid 
and  86%  for  the  Indirect  Fire  Area.  In  the  both  areas,  the  weakest  performance  was  for 
the  1 05mm  projectiles.  By  comparison,  the  TEMTADS  5x5  array  performance  statistics 
were  97%  and  92%  for  the  Blind  Grid  and  the  Indirect  Fire  Areas,  respectively. 

Table  7-2  -  TEMTADS  Hand-Held  Sensor  Blind  Grid  Test  Area  Results 


disc 

Ad 

All  Types 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.96 

0.90 

0.97 

1.00 

Oto  4D 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

1.00 

1.00 

1.00 

1.00 

8D  to  12D 

0.56 

0.50 

0.00 

1.00 
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Table  7-3  -  TEMTADS  Hand-Held  Sensor  Blind  Grid  Test  Area  Results 


j\  disc 
^fp 

All  Masses 

0  to  0.25  kg 

>0.25  to  1  kg 

>1  to  10  kg 

All  Depths 

0.07 

0.03 

0.02 

0.50 

0  to  0.15m 

0.06 

0.04 

0.02 

0.50 

0.15  to  0.3m 

0.13 

0.00 

0.00 

0.50 

0.3  to  0.6m 

N/A 

N/A 

N/A 

N/A 

Table  7-4  -  TEMTADS  MP  2x2  Cart  Blind  Grid  Test  Area  Pd'*'""  Results 


disc 

Ad 

All  Types 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.94 

0.90 

0.97 

0.97 

0  to  4D 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

0.90 

0.50 

1.00 

0.95 

8D  to  12D 

0.78 

0.83 

0.00 

1.00 

Table  7-5  -  TEMTADS  MP  2x2  Cart  Blind  Grid  Test  Area  Pfp'*'""  Results 


n  disc 

Pfp 

All  Masses 

0  to  0.25  kg 

>0.25  to  1  kg 

>1  to  10  kg 

All  Depths 

0.40 

0.64 

0.14 

0.40 

0  to  0.15m 

0.41 

0.64 

0.11 

0.67 

0.15  to  0.3m 

0.31 

0.60 

0.29 

0.00 

0.3  to  0.6m 

N/A 

N/A 

N/A 

N/A 

Table  7-6  -  TEMTADS  5x5  Array  Blind  Grid  Test  Area  Pd*®*^  Results 


jy  disc 

Ad 

^EJSSSf^ 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.97 

0.93 

0.97 

1.00 

0  to  4D 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

1.00 

1.00 

1.00 

1.00 

8D  to  12D 

0.67 

0.67 

0.00 

1.00 

Table  7-7  -  TEMTADS  5x5  Array  Blind  Grid  Test  Area  Pfp‘^'®‘^  Results 


jy  disc 

Pfp 

All 

Masses 

0  to  0.25 
kg 

>0.25  to  1 
kg 

>1  to  10 
kg 

All  Depths 

0.01 

0.02 

0.00 

0.00 

0  to  0.15m 

0.01 

0.02 

0.00 

0.00 

0.15  to 

0.3m 

0.00 

0.00 

0.00 

0.00 

N/A 

N/A 

N/A 

N/A 
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Table  7-8  -  TEMTADS  MP  2x2  Cart  Indirect  Fire  Test  Area  Results 


disc 

Ad 

All  Types 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.86 

0.84 

0.88 

0.88 

By  Density 

High 

0.81 

.085 

0.77 

0.80 

Medium 

0.89 

0.87 

0.89 

0.90 

Eow 

0.89 

0.81 

0.94 

0.91 

By  Depth 

0  to  4D 

0.93 

0.87 

1.00 

0.95 

4D  to  8D 

0.79 

0.84 

0.77 

0.71 

8D  to  12D 

0.72 

0.50 

0.89 

0.67 

Table  7-9  -  TEMTADS  MP  2x2  Cart  Indirect  Fire  Test  Area  Pfp‘^‘®‘^  Results 


n  disc 

All  Masses 

0  to  0.25  kg 

>0.25  to  1  kg 

>1  to  10  kg 

All  Depths 

0.21 

0.26 

0.14 

0.22 

0  to  0.15m 

0.21 

0.25 

0.13 

0.33 

0.15  to  0.3m 

0.20 

0.31 

0.19 

0.12 

0.3  to  0.6m 

0.17 

1.00 

0.17 

0.00 

Table  7-10  -  TEMTADS  5x5  Array  Indirect  Fire  Test  Area  Pd**‘^  Results 


disc 

Ad 

H!!I35!I3 

105-mm 

81/60mm 

37/25-mm 

Munitions 

Scores 

0.92 

0.93 

0.93 

0.91 

By 

Density 

High 

0.88 

0.92 

0.91 

0.80 

Medium 

0.94 

0.97 

0.89 

0.97 

Eow 

0.94 

0.90 

0.97 

0.94 

By  Depth 

0  to  4D 

0.96 

0.94 

0.97 

0.97 

4D  to  8D 

0.92 

0.94 

0.92 

0.86 

8D  to  12D 

0.72 

0.75 

0.78 

0.67 
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Table  7-1 1  -  TEMTADS  5x5  Array  Indirect  Fire  Test  Area  Ffp***^  Results 


n  disc 

Rfp 

All 

Masses 

0  to  0.25 
kg 

>0.25  to  1 
kg 

>1  to  10 
kg 

All  Depths 

0.04 

0.03 

0.02 

0.11 

0  to  0.15m 

0.04 

0.04 

0.02 

0.13 

0.15  to 

0.3m 

0.04 

0.00 

0.06 

0.06 

0.3  to 

0.6m 

0.08 

0.00 

0.00 

0.20 

Discrimination  Efficiency  (E)  and  Ealse  Positive  Rejection  Rate  (Rfp)  measure  the 
effectiveness  of  the  discrimination  stage  proeessing.  The  goal  of  diserimination  is  to 
retain  the  greatest  number  of  munitions  deteetions  from  the  anomaly  list,  while  rejeeting 
the  maximum  number  of  anomalies  arising  from  non-munitions  items.  Efficiency 
measures  the  fraction  of  detected  munitions  retained  after  diserimination,  while  the 
rejection  rate  measures  the  fraction  of  false  alarms  rejeeted.  The  measures  are  defined 
relative  to  the  number  of  munitions  items  or  the  number  of  clutter  items  that  were 
aetually  deteeted  by  the  sensor. 

The  HH  sensor  results  for  the  Blind  Grid  Area  are  summarized  in  Table  7-12,  from  Table 
7a  of  Referenee  13.  The  MP  system  results  for  the  Blind  Grid  and  Indirect  Eire  Test 
Areas  are  summarized  in  Table  7-13  and  Table  7-15,  from  Tables  7a  and  7c  of  Reference 
14.  Performance  levels  are  shown  at  two  speeifie  operating  points  on  the  ROC  curve: 
one  at  the  point  where  no  decrease  in  Pd  is  incurred  and  the  other  at  the  operator-selected 
operating  point  or  threshold.  Eor  reference,  the  results  for  the  TEMTADS  5x5  array 
demonstration  are  given  in  Table  7-14  and  Table  7-16. 

Eor  the  HH  sensor,  this  objective  was  successfully  met,  with  99%  of  emplaced  munitions 
items  detected  at  the  operating  point  with  a  corresponding  false  positive  rejeetion  rate  of 
93%.  The  MP  system  came  very  close  to  meeting  this  objective.  97%  of  the  emplaced 
munitions  were  eorreetly  elassified  at  our  seleeted  operating  point,  with  a  corresponding 
false  positive  rejeetion  rate  of  53%.  In  the  Indirect  Eire  Area,  94%  of  the  emplaeed 
munitions  were  eorreetly  classified,  with  a  corresponding  false  positive  rejection  rate  was 
54%.  Eor  referenee,  the  TEMTADS  5x5  array  results  for  the  Blind  Grid  were  99%  of 
emplaced  munitions  items  were  deteeted  at  the  operating  point  with  a  corresponding  false 
positive  rejection  rate  of  99%.  Eor  the  Indireet  Eire  Area,  the  pereentages  were  98%  and 
92%,  respeetively. 


Table  7-12  -  TEMTADS  Hand-Held  Sensor  Blind  Grid 
Test  Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

At  Operating  Point 

0.99 

0.93 

With  No  Eoss  of  Pd 

1.00 

0.40 
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Table  7-13  -  TEMTADS  MP  2x2  Cart  Blind  Grid  Test 
Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

At  Operating  Point 

0.97 

0.53 

With  No  Eoss  of  Pd 

1.00 

0.15 

Table  7-14  -  TEMTADS  5x5  Array  Blind  Grid  Test  Area 
Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

At  Operating  Point 

0.99 

0.99 

With  No  Eoss  of  Pd 

1.00 

0.95 

Table  7-15  -  TEMTADS  MP  2x2  Cart  Indirect  Fire  Test 
Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

At  Operating  Point 

0.94 

0.54 

With  No  Eoss  of  Pd 

1.00 

0.01 

Table  7-16  -  TEMTADS  5x5  Array  Indirect  Fire  Test 
Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

At  Operating  Point 

0.98 

0.92 

With  No  Eoss  of  Pd 

1.00 

0.58 

7.2  OBJECTIVE:  CUED  PRODUCTION  RATE 

Even  if  the  performanee  of  the  teehnologies  on  the  metries  above  was  satisfactory,  there 
remain  economic  metrics  to  consider.  Survey  efficiency  is  the  metric  that  was  tracked  in 
these  demonstrations. 

7.2.1  Metric 

For  cued  data  collection,  the  metric  is  the  number  of  anomalies  investigated  per  day 
during  each  demonstration.  Combined  with  the  daily  operating  cost  of  the  technology, 
these  values  give  the  per-anomaly  cost  of  operating  each  technology. 

7.2.2  Data  Requirements 

Productivity  was  determined  from  a  review  of  the  demonstration  field  logs. 
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7.2.3  Success  Criteria 


Given  the  cued  data-collection  methodology  used  for  these  demonstrations,  this  objective 
was  considered  successfully  met  if  the  production  rates  were  at  least  50  and  200 
anomalies  per  day  for  the  HH  sensor  and  the  MP  system,  respectively. 

7.2.4  Results 

This  objective  was  successfully  met  for  both  demonstrated  systems. 

For  the  HH  sensor,  404  target  measurements  were  made  over  the  course  of  six  field  days 
for  an  average  of  67.3  targets/day.  These  values  include  any  necessary  reacquisitions. 
The  lowest  daily  production  rate  was  42  targets/day  and  occurred  on  the  last  day  of  a 
work  week. 

For  the  MP  system,  1,073  target  measurements  were  made  over  the  course  of  four  field 
days  for  an  average  of  268.3  targets/day.  These  values  include  any  necessary 
reacquisitions.  Only  14  targets  were  measured  on  the  fourth  day,  with  the  remainder  of 
the  day  spent  packing  equipment  and  demobilization.  The  average  production  rate  for  the 
three  full  days  was  353  targets/day. 

7.3  OBJECTIVE:  ANALYSIS  TIME 

Another  component  of  demonstration  costs  was  the  amount  of  analyst  time  required  for 
data  analysis.  We  tracked  the  near-real-time  analysis  time  for  these  demonstrations. 

7.3.1  Metric 

The  time  required  for  inversion  and  classification  per  anomaly  was  the  metric  for  this 
objective 

7.3.2  Data  Requirements 

Analysis  time  was  determined  from  a  review  of  the  data  analysis  logs. 

7.3.3  Success  Criteria 

Since  these  were  the  first  formal  demonstrations  of  these  technologies,  the  objective  was 
considered  successfully  met  if  the  average  inversion  and  classification  time  was  less  than 
15  min  per  anomaly. 

7.3.4  Results 

This  Objective  was  successfully  met.  For  the  HH  sensor,  several  minutes  were  required 
to  invert  the  data  and  generate  the  data  quality  review  and  inversion  results  graphics  on 
our  field  laptop  computer.  If  any  data  cleanup  /  editing  was  required  for  a  particular  data 
collection,  the  process  would  add  several  minutes  of  processing  time.  The  average 
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analysis  time  amounted  to  10  minutes  per  anomaly.  For  the  MP  system,  the  average 
inversion  time  per  target  was  approximately  30  seconds  on  our  field  laptop  computer. 

This  time  includes  inverted  both  data  sets  individually  and  then  jointly,  so  that  all  three 
sets  of  results  can  be  evaluated.  Including  this,  the  average  analysis  time  amounted  to  5 
minutes  per  anomaly.  As  a  result  of  lessons  learned  from  this  undertaking,  we  expect  the 
average  analysis  time  for  future  field  runs  to  be  less  than  that  obtained  here. 

7.4  OBJECTIVE:  EASE  OF  USE 

This  objective  represents  an  opportunity  for  all  parties  involved  in  the  data  collection 
process,  especially  the  data  collection  team,  to  provide  feedback  in  areas  where  the 
process  could  be  improved. 

7.4.1  Data  Requirements 

Discussions  with  the  entire  field  team  and  other  observations  were  used. 

7.4.2  Results 

This  Objective  was  successfully  met.  Based  on  operator  feedback,  there  were  no 
significant  limitations  to  the  efficient  use  of  either  system  in  the  field.  Several 
suggestions  were  made  for  additional  improvements  to  the  data  collection  software.  They 
are  in  the  process  of  being  incorporated. 

7.5  OBJECTIVE:  RELIABILITY 

This  objective  captures  the  readiness  of  the  system  for  live  site  demonstrations  as  an 
integrated  system. 

7.5.1  Data  Requirements 

The  number  of  operational  hours  per  day  and  the  frequency  of  significant  technical  issues 
were  collected  from  the  demonstration  field  logs. 

7.5.2  Results 

This  objective  was  successfully  met  for  both  systems.  No  significant  downtime  was 
caused  by  system  failures.  One  issue  was  uncovered  during  the  MP  system  testing  in 
August.  The  data  collection  electronics  were  originally  designed  for  the  HH  sensor  and 
expanded  to  operate  the  MP  system  during  construction.  The  additional  cabling  and 
electronics  decreased  the  air  circulation  and  increased  the  heat  loading  of  the  system. 
Additionally,  two  DC/DC  power  supplies  in  the  transmitter  circuit  were  improperly 
configured  such  that  they  operated  at  a  significantly  increased  temperature.  These  issues, 
taken  together,  lead  to  transmitter  instabilities.  Hourly  rotation  of  ice  packs  placed  on  the 
electronics  cover  alleviated  the  problem.  With  the  increased  data  collection  tempo  for  the 
HH  sensor  (40  measurements  per  anomaly,  versus  8  for  the  MP  system),  the  situation  was 
only  further  aggravated  to  the  point  that  the  ice  packs  were  necessary  during  the  HH 
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sensor  demonstration  in  October,  2010  where  the  ambient  temperature  was  only  50  “F. 
The  ice  packs  were  rotated  during  data  download  and  battery  swap  periods,  so  they  did 
not  impact  production  rates.  Since  these  demonstrations,  these  issues  have  been 
addressed  and  ice  packs  are  no  longer  required. 

7.6  REMINGTON  WOODS  SURVEY  DATA  SUMMARY 

7,6,1  Remington  Woods,  2008 

A  simple  fiberglass  cart  carrying  four  of  the  MTADS  TEM  coils  was  assembled  to 
illustrate  the  MP  system  concept  at  the  SERDP/ESTCP/NAOC  Technology  Transfer 
Workshop  in  July  of  2008.  In  October  this  "pre-prototype"  array  was  tested  at  DuPont's 
Remington  Woods  site  in  Bridgeport,  CT.  These  tests  used  the  full  TEMTADS  5x5 
electronics  package  tethered  to  a  rugged  notebook  computer  to  control  the  MP  system. 
The  electronics  and  batteries  were  carried  in  a  garden  cart  (Figure  7-1,  left). 


Figure  7-1  -  Pre-prototype  TEMADS  MP  2x2  Cart  testing  at  Remington  Woods  in  2008 
(left)  and  2009  (right). 

Data  collection  included; 

E  Test  stand  measurements  of  simulants  of  the  targets  of  interest  (37  mm,  47  mm, 

57  mm,  66  mm,  75  mm  and  105  mm  projectiles), 

2.  A  test  field  seeded  with  a  variety  of  ordnance  simulants  and  representative  clutter 
items,  and 

3.  Portions  of  a  35  acre  section  of  the  live  site  cleanup  area. 

Results  from  the  test  field  were  mixed.  Ten  of  eleven  ordnance  items  and  pipe  sections 
were  correctly  identified  as  ordnance.  The  remaining  ordnance-like  item  (a  IV2"  diameter 
by  6"  long  pipe  section)  was  classified  as  low  confidence  clutter  on  the  basis  of  a  poor 
library  match  metric,  although  there  was  a  visual  match  of  the  inverted  polarizabilities  to 
those  for  a  47  mm  simulant  from  the  test  stand  measurements.  One  clutter  item  (a  4" 
piece  of  sheet  steel)  was  incorrectly  classified  as  low  confidence  ordnance.  Thirteen 
clutter  items  were  correctly  classified,  but  inversions  of  the  data  for  the  remaining 
nineteen  of  the  forty-four  targets  in  the  test  field  produced  unphysical  results  that  could 
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not  be  used  for  classifieation,  and  were  classified  as  "can't  analyze".  Those  targets 
included  nails,  50  caliber  slugs,  "magnetic  rocks",  wire,  small  pieces  of  sheet  metal  and 
banding. 

Data  were  collected  on  100  anomalies  in  the  live  site  area  before  recurrent  fiat  tires  on  the 
electronics  cart  due  to  punctures  by  green  briar  thorns  finally  called  a  halt  to  the 
operation.  On  excavation,  four  of  these  anomalies  were  found  to  be  due  to  ordnance 
items  (intact  37  mm  projectiles).  These  had  been  classified  as  high  confidence  ordnance. 
Two  other  anomalies  had  been  classified  as  low  confidence  ordnance,  but  turned  out  to 
be  a  six  inch  long  gate  hinge  and  a  piece  of  chain.  Fifteen  of  the  anomalies  could  not  be 
classified  ("can't  analyze").  All  of  the  79  anomalies  classified  as  clutter  (3  with  high 
confidence  and  76  with  low  confidence)  were  found  to  be  clutter  items. 

Although  all  of  the  ordnance  items  in  the  live  site  data  had  been  properly  identified,  the 
large  fraction  (93%)  of  anomalies  that  had  low  confidence  or  "can't  analyze" 
classifications  was  disappointing.  Realistically,  only  high  confidence  clutter  anomalies 
can  be  left  un-dug  -  three  targets  out  of  100! 

Subsequent  analysis  and  testing  at  Blossom  Point  revealed  that  certain  target  locations 
and  orientations  relative  to  the  symmetry  planes  of  the  MP  system  could  produce 
spurious  inversion  results.  In  order  to  feed  more  information  to  the  inversion  we  decided 
to  take  two  measurements  over  each  anomaly  -  one  over  the  flag  and  another  20  cm  past 
the  flag.  Rather  than  attempt  to  precisely  control  the  measurement  progression,  the  actual 
(as  opposed  to  the  nominal  20  cm)  separation  between  the  measurements  was  included  as 
another  parameter  to  be  determined  in  the  inversion.  Controlled  tests  indicated  that  this 
two-step  measurement  procedure  generally  produced  more  consistent  inversion  results 
than  a  single  shot  over  the  target. 

7,6.2  Remington  Woods,  2009 

The  backpack  electronics  package  for  the  MP  system,  which  was  shared  with  the  MR- 
200807  hand-held  TEM  sensor,  was  completed  in  2009.  The  2008  pre-prototype  array 
cart,  now  with  the  backpack  electronics  but  still  controlled  using  the  tethered  notebook 
computer,  was  again  tested  at  Remington  Woods  in  August,  2009  (Figure  7-1,  right). 

The  munitions  cleanup  practice  at  Remington  Woods,  which  has  been  approved  by  the 
EPA  for  the  site,  involves  an  initial  EM61-MK2  survey  followed  by  cued  ID  of  potential 
UXO  contacts  with  a  handheld  EM61-HH  using  procedures  developed  in  ESTCP  project 
MR-200108.  Roughly  one  half  of  the  original  contacts  are  typically  ruled  out  as  possible 
UXO  items  on  the  basis  of  their  EM61  and/or  EM61-HH  response  characteristics.  These 
are  referred  to  as  category  3  contacts.  Ten  percent  of  the  category  3  contacts  are 
excavated  for  quality  control  purposes.  Eor  about  one  third  of  the  initial  contacts,  factors 
such  as  overlapping  signatures,  weak  signal  levels,  etc.  make  it  impossible  to  reliably 
classify  the  target.  These  are  referred  to  as  category  2  contacts,  and  all  of  these  are 
excavated  by  default.  The  remaining  15-20%  are  categorized  as  potential  UXO  (category 
1  contacts)  based  on  the  cued  EM61-HH  analysis.  Typically,  only  a  small  fraction  of 
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these  are  actually  UXO.  We  sampled  a  selection  of  category  1,  2  and  3  contacts  on  a 
2714  acre  parcel  that  that  had  gone  through  the  EM61  survey  and  EM61-HH  cued  ID 
process  that  summer  and  was  to  be  dug  later  that  year. 

A  total  of  7 1 1  flagged  anomalies  were  interrogated  with  the  MP  system  using  the  two- 
step  procedure  over  the  course  of  214  days:  226  category  1  anomalies,  367  category  2's 
and  117  category  3's.  681  of  the  anomalies  visited  with  the  2x2  were  later  excavated, 
including  all  of  the  category  1  and  2  anomalies  and  88  of  the  category  3  anomalies.  We 
used  the  same  three  categories  (likely  UXO,  can't  tell,  likely  clutter)  to  rank  the  MP 
system  TEM  results,  and  the  classification  and  dig  results  are  summarized  in  Table  7-17. 
23  of  the  681  excavated  anomalies  were  due  to  UXO  items  (37  mm,  47  mm  and  57  mm 
projectiles),  and  528  were  due  to  clutter  (including  27  pieces  of  exploded  or  broken 
ordnance  items).  130  had  no  target  or  were  identified  as  ash  or  clinker  that  had  been 
dumped  there. 

Table  7-17  -  Classification  and  dig  results  summary  for  the  2009  Remington  Woods  test 


61HH 

Category 

2x2  Category 

Dig  Results 

1 

2 

3 

total 

UXO 

none* 

clutter 

total 

1 

40 

102 

84 

226 

17 

2 

207 

226 

2 

10 

326 

31 

367 

6 

128 

233 

367 

3 

0 

69 

48 

117 

0 

0 

88 

88 

total 

50 

497 

163 

710 

23 

130 

528 

681 

*includes  "ash/clinker" 


The  principal  results  of  the  2009  Remington  Woods  test  were  that  the  MP  system: 

1.  Correctly  re-classified  84  of  the  226  EM61-HH  category  1  (likely  UXO)  contacts 
as  clutter. 

2.  Re-classified  a  pair  of  the  EM61-HH  category  2  (can't  tell)  contacts  which 
happened  to  be  UXO  as  category  3  (likely  clutter).  One  was  a  cluster  of  three 
UXO  items,  the  other  a  single  37  mm  projectile. 

3.  Re-classified  a  37  mm  projectile  as  likely  UXO  that  had  been  "can't  tell"  with  the 
EM61-HH. 

4.  Ended  up  with  more  category  2  (can't  tell)  anomalies  (497  of  712)  than  the  EM61- 
HH  (367),  more  than  enough  to  offset  the  number  of  clutter  items  shifted  from 
EM61-HH  category  1  to  MP  system  category  3. 

All-in-all,  the  MP  system  as  tested  did  not  perform  as  well  as  the  EM61-HH  cued  ID 
approach  that  is  currently  used  at  Remington  Woods.  Retrospective  analysis  indicates 
that  the  missed  ordnance  is  more  a  failing  of  the  classification  algorithm  than  the  sensor 
itself.  The  real  problem  is  that  almost  half  (171  of  344)  of  the  anomalies  that  had  strong 
enough  response  to  be  classified  as  ordnance  or  clutter  using  the  EM61-HH  could  not  be 
classified  using  the  MP  system. 
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7.7  SPRING  VALLEY  SURVEY  DATA  SUMMARY 


The  pre-prototype  MP  system,  eonfigured  as  in  the  2009  Remington  Woods  test,  was 
tested  at  the  Dalecarlia  Woods  site  in  Spring  Valley  in  May,  2010.  107  flagged 
anomalies  were  interrogated  in  advance  of  the  dig  team.  Ground  truth  was  later  provided 
for  102  of  the  targets. 

The  results  were  not  very  informative.  62  of  the  anomalies  had  no  target  or  appeared  to 
be  magnetic  rocks  or  soil.  The  remaining  40  anomalies  were  due  to  various  clutter  items 
(horseshoes,  spikes,  wire,  scrap,  etc.).  Of  these,  27  were  found  at  an  offset  of  more  than 
40  cm  from  the  flag.  No  targets  of  interest  (75  mm  chemical  rounds,  4-in  Stokes  mortars 
or  Livens  projectiles)  were  found.  82  of  the  anomalies  interrogated  with  the  MP  system 
could  either  not  be  analyzed  or  not  be  classified.  For  most  of  these  either  there  was  no 
target  or  the  target  was  at  the  edge  of  the  array.  The  remaining  25  were  correctly 
classified  as  clutter.  At  the  time  of  the  demonstration,  target  locations  had  only  been 
verified  by  reacquisition  for  one  of  the  three  grids  to  be  investigated.  The  flags  in  the 
other  two  grids  had  not  been  reacquired  as  planned.  It  was  later  determined  that  for  the 
positions  in  the  grid  that  had  been  reacquired,  standard  operating  procedures  on  site  were 
not  to  move  the  flag  to  the  reacquired  position  but  rather  note  it  in  a  field  log.  This  was 
unknown  to  the  field  team  at  the  time  and  the  offsets  were  not  provided  during  the 
demonstration. 

7.8  DATA  ANALYSIS  IN  SUPPORT  OF  UPGRADING  EMI  SENSORS  TO  TRI- 
AXIAL  RECEIVERS  FOR  2X2  MP  CART  SYSTEM 

As  was  seen  in  earlier  Sections,  particularly  Sections  7.1,  7.6,  and  7.7,  the  performance  of 
the  MP  system  has  been  disappointing  to  date.  A  comparison  of  results  from  the 
Calibration  Area  at  APG  is  revealing  as  to  why. 

Calibration  lanes  F  through  K  contain  six  each  of  25  mm  and  37  mm  projectiles,  60  mm 
and  81  mm  mortars,  105  mm  HEAT  rounds  and  105  mm  projectiles.  The  TEMTADS 
5x5  array  correctly  classified  31  of  these  36  targets  using  an  automated  classification 
procedure.  Of  those  targets  correctly  classified  by  the  5x5,  only  17  (55%)  were  correctly 
classified  by  the  MP  system  using  a  similar  procedure.  Signal  to  noise  ratios  (SNRs)  for 
the  MP  system  and  5x5  array  do  not  appear  to  be  sufficiently  different  to  account  for  the 
difference  in  performance  between  the  two  systems.  Eigure  7-2(a)  compares  peak  signals 
measured  with  the  MP  system  with  the  corresponding  peak  signals  from  the  5x5  array  for 
the  calibration  targets  properly  classified  by  the  5x5.  The  symbols  are  color  coded  to 
indicate  MP  system  classification  performance.  A  solid  black  symbol  means  that  the 
target  was  correctly  classified  as  UXO,  and  that  inversion  of  the  data  produced 
polarizabilities  that  matched  those  expected  for  the  target  type.  Open  symbols  represent 
targets  that  would  have  been  misclassified  as  clutter.  The  shaded  gray  symbols  are  used 
for  ambiguous  results,  basically  "can't  tell".  The  median  of  the  ratios  of  MP  system  peak 
signal  to  5x5  array  peak  signal  is  43%,  and  there  is  no  significant  difference  in  this  ratio 
between  the  targets  correctly  classified  by  the  MP  system  and  the  others.  While  on  the 
calibration  grid,  RMS  noise  levels  were  a  bit  higher  for  the  5x5  array  than  the  MP  system 
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(2.57  mV  vs.  1.56  mV),  so  SNR  values  for  the  MP  system  are  only  slightly  smaller  (by 
about  30%)  than  corresponding  SNR  values  for  the  5x5  array.  While  this  may  account 
for  some  of  the  performance  differential,  it  seems  unlikely  that  it  can  account  for  all,  or 
even  most  of  it.  Figure  7-2(b)  shows  the  peak  MP  system  signals  as  a  function  of  target 
depth.  Symbols  are  color  coded  as  in  Figure  7-2(a).  Although  targets  with  the  strongest 
signals  tend  to  be  correctly  classified  while  those  with  the  weakest  signals  tend  to  be 
incorrectly  classified,  there  is  considerable  overlap  in  the  middle.  Indeed,  median  values 
for  the  two  groups  are  basically  the  same  (25.0  mV  and  25.8  mV,  respectively).  The 
main  difference  is  that  the  deeper  targets  are  the  ones  more  likely  to  be  misclassified. 

The  median  depth  of  the  17  targets  correctly  classified  by  the  MP  system  was  32  cm, 
compared  to  53  cm  for  the  other  14. 
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Figure  7-2  -  (a)  MP  system  array  peak  signals  vs.  5x5  array  peak  signals  for  calibration  targets 
correctly  classified  by  the  5x5  array.  Symbol  colors  indicate  MP  system  classification  status,  (b) 
Distribution  of  MP  system  peak  signals  and  target  depths. 

Classification  is  based  on  comparing  principal  axis  polarizabilities  estimated  from  data 
collected  over  the  target  with  those  expected  for  targets  of  interest,  and  good 
classification  performance  can  only  realized  only  if  the  polarizabilities  can  be  estimated 
accurately  from  the  data.  We  use  a  standard  dipole  inversion  procedure  to  estimate  the 
principal  axis  polarizabilities.  This  involves  searching  out  the  unknown  target  location, 
orientation  and  polarizabilities  which  minimize  the  difference  between  the  signal 
predicted  by  the  dipole  model  and  the  measured  data.  The  metric  to  be  minimized  is  the 
squared  error 
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where  S  is  the  measured  signal  vector  and  S'  is  the  signal  vector  predicted  by  the  dipole 
model  for  a  set  of  signal  parameters.  How  well  the  parameters  can  be  estimated  depends 
on  the  noise  in  the  measurements  and  the  shape  of  the  error  surface  (e'  as  a  function  of 
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the  target  parameters).  The  error  in  estimating  some  parameter  depends  on  the 
curvature  C  of  the  error  surface  in  the  direction  as 


Sd  =  ^ 

Vc 

where  cr„  is  the  RMS  noise  level  normalized  by  the  signal  strength  as  in  the  equation  for 
e^.  At  a  given  noise  level,  a  sensor  which  produces  an  error  surface  with  a  sharp 
minimum  is  better  able  to  constrain  uncertainty  in  the  target  parameter  than  one  which 
has  a  broad,  flat  region  around  the  minimum  error. 

The  shape  of  the  error  surface  depends  on  both  what  the  sensor  is  measuring  (i.e.,  the 
target  parameters)  and  how  it  is  doing  the  measuring  (data  density  and  extent,  transmit 
and  receive  coil  configurations,  etc.).  Unraveling  the  various  effects  can  be  fairly 
involved,  but  a  simple  example  serves  to  illustrate  the  basic  difference  between  the  MP 
system  (with  stepped  measurement)  and  the  5x5  array.  Figure  7-3  shows  cuts  through 
their  error  surfaces  as  functions  of  horizontal  distance  from  the  target  location  along  the 
minimum  curvature  direction.  All  other  parameters  are  fixed  at  their  true  values.  The 
target  is  axially  symmetric  with  314  to  1  polarizability  ratio  and  is  directly  under  the 
array,  aligned  with  long  axis  horizontal  and  parallel  to  the  cross-track  direction  (i.e., 
perpendicular  to  the  20  cm  step  for  the  MP  system).  The  different  plots  are  for  different 
target  distances  below  the  sensors  as  indicated.  For  a  target  at  25  cm  (on  the  surface  for 
the  MP  system,  whose  sensors  ride  27  cm  above  the  ground),  the  error  cuts  are  similar. 
For  progressively  deeper  targets  the  MP  system  error  surface  broadens  out  more  and 
more  relative  to  the  error  surface  for  the  5x5  array.  The  chain-dashed  curves  show  what 
happens  if  the  standard  single  axis  MP  system  receiver  coils  are  replaced  with  three 
component  vector  receivers,  and  we  forego  the  second  (stepped)  measurement.  The 
additional  information  from  the  horizontal  components  of  the  induced  field  at  the 
receivers  is  able  to  better  constrain  the  inversion,  and  the  error  surface  is  sharpened 
significantly  for  deeper  targets. 

The  pre -prototype  MP  system  has  proven  to  be  very  well  suited  for  cued  identification  in 
areas  inaccessible  to  the  vehicle-towed  TEMTADS  5x5  array,  achieving  production  rates 
of  lOO’s  of  targets  per  day  in  challenging  environments.  However,  its  classification 
performance  has  been  disappointing.  Too  many  targets  cannot  be  classified.  In  order  to 
improve  classification  performance  to  levels  approaching  that  of  the  full  TEMTADS  5x5 
system  we  recommended  replacing  the  single  axis  receive  coils  in  the  MP  system  with 
tri-axial  receiver  coils  in  the  final  demonstration  system.  Eigure  7-4(a)  shows  one  of  the 
10  cm  square  tri-axial  receiver  cubes  used  in  MetalMapper  (MR-200603).  The  current 
MP  system  coil,  shown  at  the  right  in  Eigure  7-4(b),  is  a  standard  TEMTADS  Rx  coil 
wound  on  a  25  cm  square  by  8  cm  high  foam  block  which  is  set  inside  the  35  cm  square 
transmit  coil  block.  The  Rx  coil  is  wound  tightly  at  the  bottom  of  the  block  nearest  the 
ground.  The  tri-axial  cubes  used  in  the  MR-201005  man-portable  vector  sensor  are  8  cm 
square  and  could  easily  replace  the  standard  receiver  coils  using  new  foam  inserts. 
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The  recommendation  to  replace  the  sensors  in  the  MP  system  from  the  original,  single¬ 
axis  receiver  TEMTADS  sensors  with  the  tri-axial  receiver  TEMTADS/3D  sensors  was 
made  to  the  ESTCP  Program  Office  in  the  winter  of  2010.  The  recommendation  was 
approved  and  the  modifications  to  the  system  made  in  early  2011. 
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Figure  7-3.  Cuts  through  error  surface  for  2x2  array  (solid  lines)  and  5x5  array 
(dashed  lines)  for  targets  25  cm,  50  cm,  75  cm  and  100  cm  below  the  array. 
Chain  dashed  curves  show  effects  of  replacing  2x2  receive  coils  with  tri-axial 
receiver  cubes. 


Figure  7-4.  (a)  MetalMapper  tri-axial  receiver  cube,  (b)  Standard  TEMTADS  transmit 
(left)  and  receiver  (right)  coils. 
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8.0  COST  ASSESSMENT 


8.1  COST  MODEL 

The  cost  elements  that  were  tracked  for  the  APG  demonstrations  are  detailed  in  Table  9-1 
and  Table  9-2.  The  provided  cost  elements  are  based  on  a  model  recently  developed  for 
cost  estimation  for  the  MP  system  at  Camp  Beale  in  201 1  [17].  The  model  assumes  a 
two-person  field  crew  and  one  data  analyst.  Production  rates  from  the  APG 
demonstration  of  these  systems  were  incorporated.  Table  9-1  contains  the  cost  model  for 
the  HH  sensor.  Table  9-2  contains  the  cost  model  for  the  MP  system.  While  neither 
system  is  currently  commercially  available,  an  estimated  daily  rental  rate  is  provided  for 
comparison  to  other  technologies.  The  rental  rate  is  based,  in  part,  on  the  costs  of  items 
purchased  in  prototype  quantities  (single  units)  and  would  presumably  decrease 
significantly  if  the  items  were  procured  at  production  quantity  levels. 

8.2  COST  DRIVERS 

Two  factors  were  expected  to  be  strong  drivers  of  cost  for  this  technology  as 
demonstrated.  The  first  is  the  number  of  anomalies  which  can  be  surveyed  per  day. 
Higher  productivity  in  data  collection  equates  to  more  anomalies  investigated  for  a  given 
period  of  time  in  the  field.  The  time  required  for  analyzing  individual  anomalies  can  be 
significantly  higher  than  for  other,  more  traditional  methods  and  could  become  a  cost 
driver  due  to  the  time  involvement.  The  thoughtful  use  of  available  automation 
techniques  for  individual  anomaly  analysis  with  operator  QC  support  can  moderate  this 
effect. 

8.3  COST  BENEFIT 

The  main  benefit  to  using  a  UXO  classification  process  is  cost-related.  The  ability  to 
reduce  the  number  of  non-hazardous  items  that  have  to  be  dug  or  have  to  be  dug  as 
presumptively-hazardous  items  directly  reduces  the  cost  of  a  remediation  effort.  The 
additional  information  for  anomaly  classification  provided  by  these  sensor  systems 
provides  additional  information  for  the  purposes  of  anomaly  classification.  If  there  is 
buy-in  from  the  stakeholders  to  use  these  techniques,  this  information  can  be  used  to 
reduce  costs. 
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9.0  IMPLEMENTATION  ISSUES 


The  goal  of  these  projects  was  to  design  and  field  units  more  amenable  to  operation  in 
more  confined  terrain  and  topology  and  smaller  tow  vehicles  /  man-portable  and 
handheld  operation  with  the  same  UXO  classification  performance  as  the  larger,  vehicle- 
towed  NRL  TEMADS.  A  second  goal  is  to  transition  these  technologies  from  being 
research  prototypes  to  use  in  the  industrial  community  where  appropriate.  The 
mechanics  of  collecting  classification-grade  EMI  data  with  these  systems  have  been 
shown  to  be  fairly  routine  in  the  research  community.  As  part  of  the  2011  ESTCP 
Munitions  Response  Eive  Site  Demonstrations,  industrial  partners  will  be  exposed  to  the 
MP  system  and  the  process  assessed.  Data  analysis  of  data  from  these  systems  remains 
somewhat  of  a  specialty,  requiring  specific  software  and  knowledge  to  conduct.  The 
successful  transition  of  the  TEMTADS  5x5  array  data  QC/analysis  process  to  the  Geosoft 
Oasis  montaj  environment  provides  a  clear  pathway  for  resolving  these  issues. 
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Table  9-1  -  TEMTADS  Hand-Held  Sensor  Tracked  Costs 


Cost  Element 

Data  Tracked 

Cost 

Data  Collection  Costs 

Component  costs  and  integration  costs 

•  Spares  and  repairs 

$3,500 

Cost  to  pack  the  array  and  equipment, 
mobilize  to  the  site,  and  return 

$9,400 

•  Personnel  required  to  pack 

1 

•  Packing  hours 

8 

Pre/Post  Survey 

•  Personnel  to  mobilize 

3 

Activities 

•  Mobilization  hours 

8 

•  Transportation  costs 

$6,000 

Cost  to  assemble  the  system,  perform 
initial  calibration  tests 

$195 

•  Personnel  required 

3 

•  Hours  required 

0.5 

Unit  cost  per  anomaly  investigated. 

This  will  be  calculated  as  daily  survey 
costs  divided  by  the  number  of 
anomalies  investigated  per  day. 

$36,90  /  anom. 

Survey  Costs 

•  Equipment  Rental  (day) 

$145 

•  Daily  calibration  (hours) 

0.2 

•  Survey  personnel  required 

2 

•  Survey  hours  per  day 

8 

•  Daily  equipment  break-down  and 

0.5 

storage  (hours) 

Processing  Costs 

$21,65  /  anom. 

Preprocessing 

Time  required  to  perform  standard  data 
clean  up  and  geophysical  data  QC. 

10  min/anom. 

Parameter 

Estimation 

Time  required  to  extract  parameters  for 
each  anomaly. 

2  min/anomaly 
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Table  9-2  -  TEMTADS  MP  2x2  Cart  Tracked  Costs 


Cost  Element 

Data  Tracked 

Cost 

Data  Collection  Costs 

Pre/Post  Survey 
Activities 

Component  costs  and  integration  costs 
•  Spares  and  repairs 

$3,500 

Cost  to  pack  the  array  and  equipment, 
mobilize  to  the  site,  and  return 

•  Personnel  required  to  pack 

•  Packing  hours 

•  Personnel  to  mobilize 

•  Mobilization  hours 

•  Transportation  costs 

$12,450 

1 

16 

3 

8 

$7,250 

Cost  to  assemble  the  system,  perform 
initial  calibration  tests 

•  Personnel  required 

•  Hours  required 

$780 

3 

2 

Survey  Costs 

Unit  cost  per  anomaly  investigated. 

This  will  be  calculated  as  daily  survey 
costs  divided  by  the  number  of 
anomalies  investigated  per  day. 

•  Equipment  Rental  (day) 

•  Daily  calibration  (hours) 

•  Survey  personnel  required 

•  Survey  hours  per  day 

•  Daily  equipment  break-down  and 
storage  (hours) 

$7,15  /  anom. 

$190 

0.5 

2 

8 

0.5 

Processing  Costs 

$10,85  /  anom. 

Preprocessing 

Time  required  to  perform  standard  data 
clean  up  and  to  merge  the  location  and 
geophysical  data. 

3  min/anomaly 

Parameter 

Estimation 

Time  required  to  extract  parameters  for 
all  anomalies. 

2  min/anomaly 
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